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1.  Summary 


This  technical  report  outlines  the  capabilities  of  Robotic  Interactive  Visualization 
Experimentation  Technology  (RIVET),  a  computer-based  simulation  environment, 
as  a  Human  Factors  platform  for  human-in- the-loop  (HITL)  human-robot 
interaction  (HRI)  experimentation.  This  system  was  originally  designed  by  the  US 
Army  Research  Laboratory’s  (ARL)  Robotics  Collaborative  Technology  Alliance 
(ARL  2012).  The  purpose  for  the  simulation  tool  was  to  provide  engineers  with  a 
means  to  test  and  debug  intelligence  and  perception  algorithms  for  autonomous 
unmanned  vehicles  prior  to  field  exercises.  Here,  we  have  adapted  the  software  to 
allow  users  to  work  cooperatively  with  unmanned  systems  to  address  the  human 
element  of  HRI.  HRI  experimentation  using  simulation  supports  the  capability  to 
assess  performance,  individual  differences,  preferences,  concerns,  and  potential 
issues  that  may  directly  or  indirectly  impact  the  design  of  the  system  for  multiple 
future  Soldier-robot  teaming  operations.  This  report  provides  a  primer  for  the 
nonprogrammer  with  example  TorqueScript  files  and  step-by-step  instructions  to 
develop  virtual  environments  (VEs)  for  HRI  experimentation.  Additional 
recommendations  for  subjective  and  objective  human  data  collection  tools  related 
to  HRI  are  discussed. 

2.  Introduction 


ARL’s  Intelligent  Systems  Enterprise  vision  is  to  enable  the  teaming  of 
autonomous  intelligent  systems  with  Soldiers  in  dynamic,  unstructured  combat 
environments,  as  well  as  in  noncombat  military  installations  and  base  operations. 
To  accomplish  this  vision  for  interdependent  Soldier-robot  teaming,  there  has  been 
a  paradigm  shift  in  robotic  research  conducted  by  ARL  from  the  current 
instantiation  of  fielded  remote-controlled  or  teleoperated  robots  to  systems  with 
increased  intelligence,  decision-making  capability,  and  autonomy  (Groom  2008; 
Chen  and  Terrence  2009;  Phillips  et  al.  2011;  Schaefer  2013).  This  type  of  teaming 
is  needed  for  future  joint,  interdependent,  network-enabled  operations. 

While  the  technological  capabilities  of  robotic  systems  are  advancing  by  the  day, 
many  of  these  new  systems  are  still  in  the  early  stages  of  research  and  development. 
In  many  cases,  areas  of  need  or  potential  use  have  been  identified,  preliminary 
requirements  have  been  created,  and  engineering  solutions  for  prototype  systems 
have  been  researched.  However,  the  human  element  must  be  considered  early  on  in 
this  design  process,  because  without  considering  human  factors,  such  as  human- 
system  interfaces,  performance,  as  well  as  trust  and  expectation,  the  potential  result 
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will  lead  to  limited  or  inappropriate  use  of  the  system  (Parasuraman  and  Riley  1997; 
Lee  and  See  2004). 

Computer-based  simulation  provides  a  safe,  economical,  and  efficient  means  to 
assess  HRI  throughout  the  robot  life  cycle.  The  following  sections  of  the 
introduction  include  a  brief  overview  of  HRI  and  HRI  metrics,  a  description  of  how 
HRI  can  be  included  into  robotic  technology  research  and  development,  and  an 
overview  of  the  benefits  of  using  simulation  for  HRI  experimentation.  This  will 
then  lead  to  a  description  and  main  purpose  of  this  report:  how  to  use  the  RIVET 
computer-based  simulation  system  for  HRI  experimentation.  A  primer  for  the 
nonprogrammer  with  example  TorqueScript  files  and  step-by-step  instructions  to 
develop  the  VE  is  provided  in  the  appendixes. 

2.1  Human-Robot  Interaction 

HRI  is  “a  field  of  study  dedicated  to  understanding,  designing,  and  evaluating 
robotic  systems  for  use  by  or  with  humans”  (Goodrich  and  Schultz  2007).  The  study 
of  HRI  is  a  relatively  new  field  (emerging  in  the  mid-1990s),  but  one  that  is  gaining 
traction  because  of  the  increase  of  robotic  technology  in  both  the  public  and 
government,  particularly  defense  applications.  As  with  any  new  field  of  study, 
researchers  are  still  developing  a  common  understanding  of  metrics  and  evaluation 
techniques.  Many  different  researchers  have  attempted  to  develop  a  common 
ground  for  understanding  and  measuring  HRI,  each  of  which  has  provided  some 
additional  insight  into  this  problem.  However,  specific  lines  of  research  have  taken 
specific  perspectives  to  experimentation  as  a  means  to  obtain  generalizable 
knowledge.  A  problem  with  this  type  of  methodology  is  that  it  results  in  context- 
specific  or  task-specific  findings  that  may  or  may  not  be  generalizable  to  other 
domains  or  applications.  In  addition,  a  number  of  measurement  techniques  exist, 
including  self-report  measures,  behavioral  measures,  task  perfonnance,  and 
psychophysiological  measures  (Bethel  et  al.  2007).  Each  type  of  measurement  has 
its  own  unique  benefits  and  limitations.  In  the  following  paragraphs,  we  briefly 
reference  some  key  findings  that  may  be  helpful  as  researchers  seek  to  better 
incorporate  HRI  assessment  into  computer-based  simulation  research. 

We  begin  by  highlighting  the  work  by  Steinfeld  et  al.  (2006),  which  attempts  to 
identify  metrics  that  can  be  used  across  robot  domains  and  applications.  They 
suggest  the  importance  of  clearly  identifying  what  they  call  the  “task-oriented” 
requirements,  instead  of  the  “task-specific”  requirements.  This  publication  is  a 
well-recognized  description  of  common  metrics  for  HRI.  The  common  task- 
oriented  metrics  identified  in  this  work  include  navigation,  perception, 
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management,  manipulation,  and  social  interaction.  The  following  bullets  list  the 
associated  measureable  items  for  each  metric. 

•  Navigation  throughout  the  environment:  effectiveness  (percent  of  tasks 
complete,  deviation  from  planned  route,  and  number  of  obstacles  avoided); 
efficiency  (time  to  complete  task,  operator  time  for  task,  and  average  time 
for  obstacle  extraction);  and  workload  (the  number  of  interventions  by  the 
user); 

•  Perception  about  objects  in  the  environment  based  on  sensor  input: 
detection  measures;  recognition  measures,  such  as  classification;  judgments 
about  the  distance,  size,  or  length  of  the  environment;  estimates  of  absolute 
and  relative  motion; 

•  Management  of  the  actions  of  humans  and  robots:  fan  out;  intervention 
response  time;  level  of  autonomy  discrepancies; 

•  Manipulation  of  the  robot  interacting  with  the  environment,  such  as 
grasping,  pushing,  or  payload  drop-off:  degree  of  mental  computations  and 
contact  errors;  and 

•  Social  interaction :  interaction  characteristics;  persuasiveness  of  the  robot; 
trust;  engagement;  compliance. 

Goodrich  and  Schultz  (2007)  identified  key  challenges  specific  to  uncertainty 
related  to  unstructured  and  extreme  environments;  risk  to  people  due  to  their 
proximity  and  vulnerability  in  the  interaction;  the  integration  of  appropriate  social 
and  emotional  aspects  of  interaction;  and  issues  with  integrating  natural  language 
into  human-robot  communication  paradigms.  They  identified  multiple  design- 
based  attributes  that  can  affect  the  interaction  of  humans  with  robots.  These 
included  the  following: 

•  The  level  and  behavior  of  autonomy:  neglect  tolerance; 

•  Nature  of  infonnation  exchanged:  interaction  time,  mental  workload, 
situation  awareness,  common  ground; 

•  Structure  of  the  team:  fan-out,  role  of  the  team; 

•  Adaptation,  learning,  and  training  of  people  and  the  robot;  and 

•  Task-shaping:  goal-directed  task  analyses,  cognitive  work  analyses,  and 
ethnographic  studies. 
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More  recently,  Weiss  et  al.  (2009)  took  a  social  approach  and  developed  an 
evaluation  framework  for  measuring  HRI.  This  framework  focuses  on  the  usability 
(e.g.,  effectiveness,  efficiency,  learnability),  social  acceptance  (e.g.,  perfonnance 
expectations,  effort  expectancy,  attitudes  toward  the  technology),  user  experience, 
and  societal  impact  (e.g.,  consequences  to  the  social  life  of  a  specific  community 
following  the  introduction  of  a  robot). 

As  robotic  design  continues  to  move  toward  interactive  collaborative  teaming  with 
people,  additional  HRI  metrics  have  emerged.  These  first  specify  the  importance  of 
communication,  highlighting  specific  elements  that  can  be  measured  from  the 
human.  For  example,  analyzing  speech  and  gesture  patterns  can  be  used  to  identify 
goal  inference,  anticipatory  action  selection,  and  congruence  of  infonnation  (Bicho 
et  al.  2010).  This  type  of  analysis  may  also  benefit  from  identifying  speech-based 
interaction  in  terms  of  both  action-only  speech  versus  action  speech  with  a 
descriptor.  Cassenti  et  al.  (2011)  found  that  people  were  more  successful  in 
directing  robots  during  a  navigation  task  if  location  labels  (e.g.,  structural  parts  of 
a  building)  were  used  and  understood  by  the  robot.  A  second  recent  area  of  study 
associated  with  HRI  metrics  has  identified  the  importance  of  measuring  situation 
awareness.  This  can  be  accomplished  by  assessing  the  stages  of  information 
processing  (i.e.,  information  acquisition,  decision  making,  and  action 
implementation),  control  allocation,  attentional  control,  multitasking,  and  task 
switching  (Chen  and  Barnes  2014). 

Most  recently,  Murphy  and  Schreckenghost  (2013)  reviewed  29  papers  regarding 
metrics  associated  with  HRI.  They  reported  that  this  is  an  area  still  being  developed 
and  understood.  Their  key  findings  related  to  the  human-robot  team  or  system  are 
reported  as  follows: 

•  Productivity  (effectiveness):  team  productivity,  task  difficulty,  and  time 
(autonomous  versus  manual  operations) 

•  Efficiency:  amount  of  effort,  human-robot  ratio,  interaction,  ratio  of 
operator  time  to  robot  time,  and  total  time  to  complete  task 

•  Reliability:  false  alarms,  flexibility,  interventions,  level  of  automation 
discrepancies,  similarity,  and  time  (intervention  response  time,  unscheduled 
manual  operations) 

•  Safety:  risk  to  people,  robot  awareness  of  people 

•  Coactivity:  cognitive  interaction,  crypsis  coefficient,  degree  of 
monotonicity,  neglect  tolerance,  plan  state,  percent  of  requests  from 
operator  and  robot,  task  allocation,  and  time  in  unscheduled  manual 
operations 
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As  can  be  seen  by  this  brief  review  of  the  HRI  domain,  there  is  a  direct  interplay 
between  use  and  design  that  should  be  further  explored  and  understood.  In  the 
following  subsection  we  begin  to  address  the  importance  of  including  HRI  into 
simulation. 

2.2  HRI  and  Computer-Based  Simulation 

According  to  Dautenhahn  (2007),  incorporating  HRI  evaluation  into  the  robot’s 
design  and  development  process  has  a  number  of  challenges  that  stem  from 
integrating  an  interdisciplinary  approach  with  technological  design.  However, 
taking  the  human  into  consideration  throughout  this  process  may  help  to  alleviate 
previously  established  issues  related  to  nonuse,  misuse,  disuse,  or  abuse  of 
developed  systems  (Parasuraman  and  Riley  1997),  leading  to  a  greater  return  on 
investment.  Additionally,  HITL  research  and  development  becomes  progressively 
more  important  as  robots  continue  to  move  into  an  integrated  team  member  role. 

Simulation  provides  the  opportunity  for  collaboration  between  Soldiers  and 
unmanned  platforms  at  multiple  levels  in  the  design  process.  This  can  help  to  guide 
the  design  process  and  provide  needed  infonnation  back  into  development.  The 
inclusion  of  simulation  into  the  design  process  is  not  a  new  concept.  For  example, 
simulation  is  a  cost-effective  strategy  often  used  throughout  the  entire  life  cycle  of 
software  development  to  identify  real-time  performance  shortfalls  and  to  develop 
risk-resolution  techniques  (Boehm  1988).  Here  we  suggest  that  the  modeling  and 
simulation  process  is  important  for  the  study  of  human  and  robot  interaction.  Loper 
(2015)  describes  the  modeling  and  simulation  process  in  the  following  steps: 

•  Establish  purpose  and  scope:  problem  statement 

•  Formulate  a  conceptual  model:  abstraction  of  the  real-world  system  under 
investigation,  goals 

•  Acquire  and  analyze  data:  requirements  gathering 

•  Develop  simulation  model  and  program:  operational  model  and  computer 
implementation 

•  Verify  and  validate  model  and  simulation:  accuracy  of  conceptual  model, 
requirements,  and  representation  of  the  real-world 

•  Design  experiments  with  specific  details:  length,  number  of 
runs/replications,  manner  of  initialization  (e.g.,  training) 

•  Execute  simulation  and  analyze  output:  estimate  measures  of  perfonnance, 
and  update  analysis  and  design  elements 
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For  the  purposes  of  this  work,  we  are  focusing  on  virtual  or  computer-based 
simulation.  More  specifically,  we  are  interested  in  the  benefits  of  HITL  simulation 
at  the  research  and  development  stages.  While  one  key  benefit  of  simulation  is  rapid 
and  systematic  development  and  assessment  of  technology,  the  process  by  which  a 
human  uses  or  understands  the  benefits  of  the  technology  can  directly  impact  the 
process  by  which  a  system  will  be  used  in  the  future.  Here,  we  suggest  that 
incorporating  HITL  simulation  experimentation  early  on  in  the  research  and 
development  process  can  benefit  the  analysis,  design,  and  development  stages  for 
robotic  technology. 

Computer-based  simulation  has  become  an  important  part  of  HITL 
experimentation,  and  provides  a  number  of  benefits,  primarily  those  related  to  cost 
and  safety  across  a  number  of  applications  areas.  More  recently,  this  type  of 
experimentation  has  been  used  to  assess  HRL  Types  and  uses  for  simulation  include 
prototype  design  and  development  (e.g.,  algorithm  testing,  sensor  design,  and 
control  interfaces),  training  and  practice  interacting  with  robotic  systems,  and 
transitions  to  real-world  robotics. 

Within  HRI,  benefits  of  simulation  include  time  and  cost  benefits  due  to  rapid  and 
repeatable  assessment  of  HRI.  The  recommendation  for  simulation  use  for  HRI 
experimentation  took  root  in  the  1980s  with  Sheridan’s  (1986)  research  into  the 
human’s  supervisory  control  of  robotic  space  systems.  While  he  recommended  that 
both  computer  simulation  and  hardware  prototype  development  are  needed  for 
design  and  development,  a  key  benefit  of  simulation  is  time  due  to  the  quick  and 
systematic  adaptations  of  the  robotic  system.  In  the  following  decade,  research 
demonstrated  cost-related  benefits  through  a  reduction  in  development  time  and 
required  financial  need.  For  example,  Michel  (1998)  demonstrated  this  cost  benefit 
through  the  development  of  the  Webots  simulator  for  robotic  prototype 
development.  This  simulator  allowed  for  the  successful  design  of  the  mechanical 
structure  of  robots,  as  well  as  the  development  of  intelligent  controllers.  The 
success  of  this  approach  was  attributable  to  the  rapid  and  repeated  testing  that  was 
not  possible  with  real-world  systems.  In  addition,  the  scarcity  and  expense  of  real- 
world  robotic  systems  also  made  simulation  a  viable  option  to  test  human 
perception,  situation  awareness,  and  teamwork,  as  seen  with  unmanned  search  and 
rescue  robots  (Nourbakhsh  et  al.  2005;  Lewis  et  al.  2007). 
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The  medical  robotics  community  has  identified  some  training  benefits  of 
integrating  HRI  principles  into  simulation  systems.  In  a  review  of  the  research, 
Kunkler  (2006)  suggested  that  the  similarities  between  computer  simulation  tools 
and  robotic  surgery  systems  (e.g.,  mechanized  feedback,  monitors  to  visualize  task, 
and  similar  computer  software  for  HRI)  provided  successful  training  opportunities 
for  physicians,  while  maintaining  personal  and  patient  safety.  In  addition, 
Basdogan  et  al.  (2004)  suggested  that  the  inclusion  of  haptic  interfaces,  rendering, 
and  recording,  as  well  as  playback  options  into  the  computer  simulation,  can 
significantly  improve  training  transfer  to  real-world  robotic  systems. 

This  goal  of  successful  transfer  from  simulation  to  real-world  HRI  is  a  common 
theme  across  all  domains  of  robotics.  One  common  practice  is  the  development  of 
compatible  source  codes  (e.g.,  robot  motion  [Davies  2000]  and  Webots  [Michel 
1998]).  For  example,  in  their  research  comparing  a  virtual  and  real-world  Pioneer 
robot,  Gerkey  et  al.  (2003)  suggested  that  while  there  is  no  guarantee  for  direct 
comparability  between  simulation  and  real-world  robotic  systems,  there  is  promise 
of  such  transfer  when  testing  new  sensors  or  control  interfaces  prior  to  hardware 
development. 

3.  RIVET  and  CARVE  for  HRI  Experimentation 

Simulation  has  opened  the  door  to  understand  and  assess  the  human,  as  well  as 
components  of  the  interaction,  early  on  in  the  design  process.  This  allows 
researchers  and  roboticists  to  begin  to  assess  the  interaction-specific  design 
elements  proactively  rather  than  retroactively.  Many  HRI  studies  have  begun  to 
rely  on  simulation  for  some  element  of  experimentation.  Many  platforms  are  used 
to  assess  interaction  with  algorithms,  such  as  planning  algorithms  (Sycara  and 
Lewis  2003)  and  for  testing  human  preferences  and  behaviors  toward  sensors 
(Hughes  and  Lewis  2004).  Simulation  is  also  used  to  study  elements  of 
communication  with  robots,  including  haptics  (Her  and  Hsu  2001),  semantic  maps 
(Nielsen  et  al.  2004),  and  bidirectional  communication  (Rickheit  and  Wachsmuth 
2008).  Individual  differences  and  elements  of  teaming  have  also  been  studied  using 
simulation.  These  include  topics  such  as  workload  and  situation  awareness 
(Parasuraman  et  al.  2003),  as  well  as  control  of  multiple  robots  (Olsen  and  Wood 
2004;  Chen  et  al.  2010).  HRI-oriented  simulation  should  also  “accurately  [reflect] 
the  range  of  available  information,  behavior,  and  user  experience  encountered  in 
actual  robot  operation”  (Lewis  et  al.  2007,  101). 
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3.1  Overview  of  RIVET 


In  this  section,  we  provide  an  overview  of  the  Robotic  Interactive  Visualization  and 
Experimentation  Technology  (RIVET)  simulation  environment.  RIVET  version 
1.0  was  built  using  the  Torque  Game  Engine  (TGE)  version  1.5.2,  designed  by 
GarageGames.  It  was  designed  by  the  ARL’s  Robotics  Collaborative  Technology 
Alliance  (RCTA)  to  allow  engineers  to  test  and  debug  intelligence  and  perception 
algorithms  for  autonomous  unmanned  vehicles  prior  to  field  exercises  (ARL  2012). 
While  field  exercises  provide  a  valuable  insight  on  the  perfonnance  of  mobility  and 
perception  algorithms,  the  amount  of  time  needed  for  field  exercises  must  be  used 
wisely.  The  cost  to  conduct  data  collection  and  testing  in  the  field  is  substantially 
more  than  time  spent  in  the  lab  conducting  testing.  This  cost  not  only  includes  a 
financial  cost,  but  also  a  time-associated  cost  in  that  it  is  difficult  to  organize  and 
coordinate  multidisciplinary  integration  sessions.  Simulation  tools  like  RIVET  can 
provide  an  essential  initial  evaluation  of  robotic  algorithms  and  concepts  in 
simulated  environments  that  can  be  varied  systematically. 

While  RIVET  was  originally  designed  to  test  and  debug  algorithms,  the  design  and 
capabilities  of  the  software  lend  themselves  to  the  development  of  HRI-specific 
missions  with  unmanned  ground  vehicles  (UGVs),  which  is  the  focus  of  the 
remainder  of  this  report.  RIVET  provides  the  underpinning  for  autonomous 
movement  that  allows  experimentation  regarding  the  interaction  of  Soldiers  and 
these  highly  technical  mobile  vehicles.  The  following  sections  describe  the 
extensive  effort  allocated  to  the  adaptation  and  advancement  of  RIVET  to  be  used 
for  HRI  experimentation  purposes. 

3.1.1  General  Capabilities 

Like  most  commercial  off-the-shelf  game  engines,  the  TGE  provides  functionality 
for  graphics,  physics,  artificial  intelligence,  lighting,  and  many  other  features.  It 
provides  all  the  necessary  attributes  and  core  functionality  to  conduct  virtual 
experiments.  Building  a  simulated  environment  involves  several  different  activities 
necessary  for  HRI  experimentation:  1)  creating  the  virtual  terrain  surface,  2)  adding 
static  features  typically  found  in  the  scene,  3)  adding  dynamic  elements,  such  as 
people  and  vehicles,  4)  producing  the  interactive  graphical  user  interfaces  (GUIs), 
5)  choosing  the  unmanned  platfonn  for  the  exercise,  6)  implementing  a  user 
interface  for  the  UGV,  and  finally,  7)  assessing  the  user  during  a  study.  Specifics 
relating  to  each  of  these  will  be  discussed  in  detail  in  Section  4  (Procedures  for 
Using  RIVET  and  HRI  Experimentation). 
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3.1.2  Hardware/Software  Considerations 

The  game  engine  adopts  a  client-server  architecture.  This  is  because  it  is  designed 
for  networking  multiple  systems  together  to  play  a  game  between  multiple  remote- 
located  users.  However,  RIVET  can  be  used  in  single-user  mode  or  multi-user 
mode  depending  on  the  problem  statement  and  research  goals.  In  single-user  mode, 
RIVET  loads  the  scene  and  then  a  user  is  able  to  access  the  world  editor,  GUI  editor, 
or  control  an  avatar  (Soldier,  vehicle,  etc.)  through  the  mission  area  using  keyboard 
or  joystick.  Multi-user  mode  allows  up  to  64  local  area  networked  users  to  join  the 
mission  as  clients.  These  clients  can  be  another  vehicle,  robot,  Soldier,  or  a  sensor 
that  is  attached  to  an  entity  in  the  simulation.  Currently,  the  choice  of  additional 
vehicle  sensors  within  RIVET  are  cameras,  laser  detection  and  ranging  (LADAR), 
and  a  custom  interface  used  to  connect  other  applications  to  the  simulation 
environment.  Sensors  can  be  local  to  the  initial  server  client  machine  or  they  can 
be  a  remote  sensor  as  a  client  on  a  different  machine.  The  key  to  detennining  which 
is  right  for  the  mission  depends  on  the  angle  of  view  for  the  sensor.  The  local  sensor 
view  is  what  the  server  camera  is  projecting.  If  a  different  camera  sensor  view  is 
needed,  a  separate  computer  running  as  a  client  will  be  used  to  connect  to  the 
vehicle,  and  then  that  graphics  processor  unit  is  used  to  develop  the  sensor  scene 
(e.g.,  rear-view  camera  for  traveling  in  reverse).  General  specifications  relating  to 
the  quality  of  the  computers,  graphics  cards,  etc.  can  be  found  in  the  RIVET 
Developer’s  Guide  (General  Dynamics  Robotics  Systems  2010,  17). 

3.1.3  RIVET  Menu  Options 

Through  the  RIVET  main  menu  (Fig.  1),  there  are  customization  options  that  are 
available  through  buttons  at  the  bottom  of  the  menu  screen.  It  is  possible  to  load  a 
customized  user  mission  (Server  computer),  join  a  mission  (Client  computer),  or 
mount  a  sensor  to  a  vehicle  or  robot  on  the  RIVET  server  computer  (Client 
computer).  In  addition,  the  configuration  menu  allows  customization  of  the  video 
screen  size,  resolution,  and  vehicles.  Appendix  A  provides  more  RIVET  menu 
options. 
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Fig.  1  RIVET  Main  Menu  GUI 

For  a  majority  of  our  FIRI-related  research,  we  need  to  have  more  human  interaction 
with  the  robot/autonomous  vehicle  than  observing  a  camera  sensor  only.  Therefore, 
the  recommended  setup  is  the  3-computer  setup  with  the  RIVET  server,  the 
additional  client  sensor,  and  the  user  interface  application  for  extra  control  and 
integration  of  the  UGV.  The  RIVET  architecture  also  allows  for  the  connection  of 
outside  applications  by  mounting  a  Basic  Operations  Layer  Transmissions  (BOLT) 
sensor  to  the  vehicle.  The  BOLT  libraries  allow  external  interfaces  to  connect  with 
the  simulation  (more  information  can  be  found  in  the  Application  Program 
Interface,  which  is  included  with  the  software).  One  such  application  is  the  Control 
for  Autonomous  Robotic  Vehicle  Experiments  (CARVE)  user  interface  application 
(discussed  in  detail  in  Section  3.2).  The  Server,  Client,  and  CARVE  Application 
computers  are  connected  through  an  Ethernet  local  area  network  (LAN),  using  a 
Gigabit  switch  (Fig.  2). 
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Fig.  2  RIVET  and  CARVE  setup  for  single  user,  remote  sensor,  or  GUI  combination 

3.2  Overview  of  CARVE 

CARVE  is  written  in  C  Sharp  (C#),  allowing  the  use  of  all  of  the  Microsoft  .NET 
libraries.  As  its  name  implies,  additional  robotic  vehicle  control  is  the  main 
functionality  of  CARVE.  For  example,  CARVE  provides  customizable  integration 
of  new  user  interface  controls  for  the  simulated  vehicle,  as  well  as  feedback 
displays  to  the  participant.  There  was  additional  work  completed  to  utilize  CARVE 
as  an  autonomous  driving  station  for  human  use  experiments.  This  included  the 
capability  to  mirror  autonomous  vehicle  behavior  through  the  integration  of 
dynamic  waypoints  (in  CARVE)  and  trigger  points  (in  RIVET).  Both  RIVET  and 
CARVE  are  customizable  in  terms  of  recording  data  from  the  vehicle  (speed, 
health,  time,  location,  etc.),  environment  (non-player  character  movement,  triggers, 
and  events),  as  well  as  user  input  (function  allocation,  time,  number  of 
interventions).  Finally,  CARVE  provides  additional  capabilities  and 
customizations  to  the  RIVET  platform  to  advance  HRI  experimentation,  including 
visual  feedback  of  vehicle  statistics,  dynamic  map  capabilities,  and  the  inclusion  of 
additional  tasks  (e.g.,  radar  detection  task).  This  added  functionality  and  control  is 
depicted  in  Fig.  3  and  allows  HRI  assessment  to  occur  early  in  development. 
Examples  of  the  CARVE  menus  are  provided  in  Appendix  B. 
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Virtual  Environment  and  Simulated  Vehicle 


User  Interface:  Controls 


Mouse 
Keyboard 
GUI  -  touchscreen 
Gamepad 

G27  Wheel  Pedals 
Joystick 

Kinect  (gestures) 
On  OffButtons 


User  Interface:  Feedback 

RIVET:  Sensor  Output 
CARVE:  Vehicle  stats 
Future:  Tablet  Display 


EStop 
SctS  peed 
Destitution 


Speed 


**Client/Seiver  Architecture 


Fig.  3  Customizable  simulation  environment  for  HRI  (RIVET,  BOLT,  CARVE) 

Additional  descriptions  and  capabilities  of  this  software,  as  well  as  specific 
modifications  for  HRI  experimentation,  will  be  discussed  through  Section  4 
(Procedures  for  Using  RIVET  for  HRI  Experimentation).  We  note  here  that 
CARVE  may  not  be  currently  set  up  to  meet  every  possible  need  for  HRI 
experimentation;  however,  additional  user  interfaces  can  be  programmed  to 
function  within  CARVE.  This  may  require  more  advanced  knowledge  of  computer 
programming,  specifically  in  C#. 

4.  Procedures  for  Using  RIVET  for  HRI  Experimentation 

The  design  and  development  of  RIVET,  and  the  associated  CARVE  application, 
are  valuable  tools  for  HRI  experimentation.  In  this  section,  we  provide  key 
descriptions  of  the  components  and  customization  capabilities  of  this  simulation 
system.  As  such,  RIVET  and  CARVE  can  have  a  wide  range  of  HRI  operations, 
from  combat-specific  operations  (e.g.,  surveillance,  cordon,  and  search)  to  base 
operations  (e.g.,  passenger  transit  using  a  self-driving  vehicle).  However,  different 
types  of  customization  require  varying  levels  of  computer  programming 
knowledge,  which  are  noted  throughout  the  following  sections.  Step-by-step 
directions  for  using  RIVET  are  then  provided  in  the  appendixes. 

4.1  Initial  Research  Definition 

Before  starting  the  mission  setup,  the  HRI  experimenter  should  have  a  research 
question,  a  set  of  robot  system  requirements,  and  an  application.  Based  on  this 
information,  it  is  beneficial  to  create  a  storyboard  of  the  scenario  or  use  case  (Preece 
et  al.  2015).  This  will  help  the  experimenter  outline  the  necessary  elements  that  will 
be  needed  in  the  development  of  the  VE.  This  may  include  the  location  (e.g.,  Middle 
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Eastern  town  versus  US  military  post),  terrain  and  weather  specifications,  potential 
risks,  stationary  and  dynamic  objects,  anticipated  or  actual  robot  capabilities  and 
behaviors,  as  well  as  the  team  goals  and  objectives.  Directions  on  how  to  create  a 
new  mission  environment  are  provided  in  Appendix  C. 

4.2  Setting  up  the  Mission 


4.2.1  Virtual  Terrain 

The  first  step  in  developing  the  VE  in  RIVET  is  to  build  the  virtual  terrain.  The 
current  setup  and  components  of  RIVET  are  such  that  a  user  with  a  basic  computer 
science  background  has  sufficient  knowledge  to  develop  the  virtual  terrain.  When 
starting  to  develop  a  new  virtual  terrain,  we  first  recommend  exploring  the  missions 
that  already  exist  in  RIVET.  Available  mission  files  can  be  accessed  through  the 
Main  Menu  (Fig.  4)  by  choosing  a  file  from  the  Loading  Mission  screen  (Fig.  5). 
This  will  allow  HRI  experimenters  the  opportunity  to  see  what  environments  have 
already  been  built  and  could  be  leveraged  for  their  needs. 


Fig.  4  Main  Menu  screen 
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Note:  The  red  box  highlights  the  areas  related  to  current  default  missions  that  have  been 
previously  created. 


Fig.  5  Load  Mission  screen 

It  is  possible  that  none  of  the  available  mission  environments  are  a  direct  match  for 
the  current  research  goal.  If  this  is  the  case,  it  is  still  possible  to  begin  with  a  base 
environment  and  customize  it  to  meet  the  specific  needs  of  the  mission.  The  ARL 
Test  World  is  a  recommended  option  to  start  to  develop  a  new  mission  file.  It  has  a 
relatively  blank  canvas,  containing  only  a  Soldier  following  a  path  and  a  small 
building.  After  entering  the  VE,  it  is  possible  to  adapt  the  terrain  to  match  the  needs 
of  the  mission.  The  terrain  is  sculpted  using  the  RIVET  terrain  editor  tools.  These 
tools  permit  changes  in  the  appearance  of  the  terrain  using  different  textures  that 
represent  rocks,  gravel,  grass,  dirt,  or  the  forest  floor.  Instructions  for  adapting  the 
terrain  are  provided  in  Appendix  D.  More  artistic  models,  such  as  grass  and  trees, 
are  also  available  by  adding  static  objects. 

4.2.2  Static  Objects 

The  TGE  allows  the  programmer  to  create  realistic  simulated  terrains  that  can  be 
textured  and  shaped  to  give  an  appearance  of  any  setting  desired.  To  illustrate  the 
complexity  that  can  be  created  with  the  editing  tool,  Fig.  6  shows  a  virtual  Middle 
Eastern  market  place  with  streets,  stalls,  and  buildings.  Each  one  of  these  items 
must  be  carefully  positioned  to  reflect  a  marketplace  in  a  developing  country. 
Depending  on  complexity,  a  scene  may  contain  hundreds  or  thousands  of  objects. 
Making  a  scene  look  professional  may  require  a  lot  of  work  and  time,  but  the 
realism  may  aid  testing  and  the  capability  to  transition  findings  to  the  field. 
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Fig.  6  Complex  world  building 

The  TGE  has  built-in  tools  that  allow  the  programmer  the  ability  to  tailor  the  scene. 
The  World  Editor  is  accessed  by  pressing  the  FI  1  key  on  the  keyboard.  Figure  7 
shows  the  marketplace  objects  augmented  with  entity  names  shown  in  white.  In  this 
view,  each  yellow  dot  represents  a  different  object,  such  as  a  building,  burned-out 
car,  and  even  the  electrical  lines.  This  is  where  objects  can  be  added,  deleted,  or 
moved  along  each  axis  (see  also  Appendix  E).  Each  object  may  also  be  rotated  in 
any  direction  along  each  axis.  Notice  that  many  of  the  objects  are  tagged  with  the 
“null”  label.  This  means  that  the  object  has  not  been  assigned  a  specific  name 
during  the  creation  of  the  scene.  Naming  each  object  is  possible  through  the 
TorqueScript  files,  the  console,  or  the  World  Editor  Inspector.  The  purpose  of  the 
World  Editor  Inspector  is  to  allow  the  evaluator  to  inspect  and  modify  parameters 
for  individual  objects.  This  is  where  the  position,  rotation,  scale,  name,  and  many 
different  dynamic  fields  can  be  changed.  These  tools  are  essential  to  building  a 
realistic  environment. 


Approved  for  public  release;  distribution  is  unlimited. 


15 


Fig.  7  Example  of  the  view  from  the  World  Editor  Inspector 

All  objects  within  the  scene  are  defined  in  a  mission  file,  which  is  saved  as  a  text 
file.  Each  object  is  listed  along  with  its  respective  parameters.  A  static  object  is 
defined  in  the  mission  file  as  shown  in  Fig.  8. 


new  TSStaticO  { 

Position  =  "-1. 72905  -129.935  -0.080005"; 
rotation  =  "0  0  -1  90"; 
scale  ="11  1"; 

shapeName  =  "~/data/shapes/Mid-East_Market/building_l.dts"; 

receiveSunLight  =  "1"; 

receiveLMLighting  =  "1"; 

useAdaptiveSelf Illumination  =  "0"; 

useCustomAmbientLighting  =  "0"; 

customAmbientSelf Illumination  =  "0"; 

customAmbientLighting  =  "0  0  0  1"; 

disableDynamicShadows  =  "0"; 

disableDynamicShadowMove  =  "0"; 

disableDynamicShadowAnimate  =  "0"; 


Fig.  8  Example  TorqueScript  for  a  static  object  located  within  the  Mission  File 


4.2.3  Dynamic  Objects:  Vehicles 

There  are  many  assets  developed  for  use  within  the  simulation  (Fig.  9).  The  vehicles 
can  move  about  using  direct  user  control  through  keyboard  or  joystick,  script-based 
path  following,  or  file-based  waypoint  following  using  either  time  on  target  or 
locations  and  velocity  data.  The  unmanned  vehicles  consist  of  entities  from  the 
following  classes: 

•  Unmanned  ground  vehicles 

•  Unmanned  air  vehicles 

•  Unmanned  surface  vehicles 
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Unmanned  underwater  vehicles 


•  Small  unmanned  ground  vehicles 


Fig.  9  Examples  of  unmanned  systems  available  within  RIVET 

Specific  missions  may  also  require  additional  non-player  vehicles  to  have 
preprogrammed  movements  throughout  the  mission  space.  Here,  the  tenn  non¬ 
player  is  included  to  differentiate  between  vehicles  that  will  be  autonomously 
moving  in  the  VE  from  player  vehicles  that  a  participant  has  an  option  to  control. 
The  current  version  of  RIVET  has  some  path-following  vehicles  already  available 
(Fig.  10).  It  is  important  to  note  that  these  dynamic  elements  used  in  a  scenario 
require  code  to  control  their  actions.  Animations  are  created  within  modeling  tools 
such  as  Autodesk  3ds  Max,  and  controlled  with  script  to  allow  movement  of 
objects.  TGE  uses  a  scripting  language,  aptly  named  TorqueScript,  to  provide  the 
control  of  these  dynamic  elements.  It  is  beyond  the  scope  of  this  report  to  provide 
an  in-depth  discussion  of  TorqueScript.  The  works  of  Maurina  (2006)  and  Finney 
(2007)  provide  more  information. 
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Fig.  10  Examples  of  path-following  vehicles  in  RIVET 


Step-by-step  directions  on  adding  a  path  and  a  path-following  vehicle  to  the  mission 
file  are  available  in  Appendix  F  and  Appendix  G,  respectively.  With  these 
directions,  the  vehicle  will  appear  by  default  when  the  mission  file  is  opened.  The 
vehicle  will  continue  on  the  path  provided.  If  the  mission  file  is  saved,  the  vehicle 
will  begin  at  the  last  saved  point  the  next  time  the  mission  file  is  opened. 

4.2.4  Dynamic  Objects:  People 

Human  characters  are  commanded  using  direct  user  control,  scripted  path 
following,  or  crowd  logic.  Similar  to  the  above  descriptions  on  path-following 
vehicles,  it  is  also  possible  to  have  path-following  people,  also  referred  to  as  non¬ 
player  characters,  or  NPCs.  These  NPCs  currently  include  civilians,  Soldiers,  and 
enemy  combatant  characters.  It  is  also  possible  to  trigger  the  NPC  to  spawn  (also 
known  as  the  creation  or  appearance  of  a  character)  on  the  path  at  a  set  point  in 
time,  as  well  as  trigger  the  NPC  to  disappear  at  a  set  time  and  location.  A  trigger  is 
“a  volume  of  space  that  initiates  script  callbacks  when  an  object  enters,  stays  inside, 
or  leaves  the  trigger’s  volume”  (http://docs.garagegames.com/torque- 
3d/reference/classTrigger.html#_details).  More  specifically,  a  trigger  causes  an 
event  to  happen.  Having  the  option  to  create  or  delete  the  NPC  allows  the 
experimenter  more  flexibility  and  consistency  between  runs  or  trials.  However,  this 
requires  additional  computer  programming,  as  well  as  additional  TorqueScript 
files.  Step-by-step  examples,  including  the  triggers  to  create  and  delete  path¬ 
following  Soldiers  are  provided  in  Appendix  H  and  Appendix  I,  respectively. 

At  times,  it  is  more  beneficial  to  use  the  crowd  logic  than  to  create  multiple  paths 
for  many  individuals.  Figure  1 1  shows  a  flurry  of  activity  among  Soldiers  and 
civilians.  These  crowds  were  generated  using  the  Crowd  Editor,  a  tool  designed  to 
add  groups  of  simulated  NPCs  to  specified  regions  of  the  VE.  Within  this  region, 
the  people  move,  interact,  and  mill  about  simulating  a  realistic  marketplace. 
Directions  for  adding  crowds  are  provided  in  Appendix  J. 
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Fig.  11  Example  of  crowd  modeling  in  RIVET 


The  RIVET  Crowd  Editor  is  a  C#  application  that  allows  a  user  to  add  groups  of 
NPCs  to  a  mission  through  a  GUI.  The  GUI  allows  for  configuring,  adjusting,  and 
controlling  these  simulated  crowds.  The  editor  is  used  to  add  crowd  entities  with  a 
specific  number  of  objects,  their  type,  and  the  percentage  of  each  character  type. 
The  editor  also  has  the  feature  where  the  user  chooses  the  number  of  destinations 
and  how  long  the  NPCs  are  idle  once  they  reach  their  destination.  The  Crowd  Editor 
GUI  is  shown  in  Fig.  12. 


[1]  Map  View  shows  all  the 
visual  crowd  data  that  is  being 
applied  to  the  selected  map. 

[2]  Add  and  Remove  is  used  to 
add  and  name  a  crowd  as  well  as 
to  delete  it. 

[3]  Crowd  List  provides  a  list  of 
all  the  crowds  created  for  the 
selected  map. 

[4]  Property  View  enables  you 
to  configure  and  edit  all  the 
aspects  for  the  specific  crowd 
selected  from  the  Crowd  List. 

[5]  Description  Window  shows 
a  description  of  the  selected 
property  in  the  Property  View. 

[6]  Tool  Box  supplies  all  the  tools 
used  to  create  and  generate  the 
crowd. 

[7]  Menu  allows  user  to  load  a 
map,  load  crowds,  or  save  a 
mission. 


Fig.  12  Crowd  Editor  GUI  in  RIVET 
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4.3  Robot  Autonomy 


Up  until  this  point,  options  and  modifications  that  are  specific  to  RIVET  have  been 
described.  However,  scenarios  using  advanced  vehicle  or  robot  autonomy  require 
the  integration  of  the  BOLT  sensor  and  the  CARVE  application.  Since  missions 
can  require  a  user  to  intervene  and  “drive”  a  vehicle,  it  was  important  to  devise  a 
way  to  communicate  with  the  vehicle  in  the  simulation.  The  BOLT  interface  was 
developed  to  allow  communication  over  the  network.  The  User  Datagram 
Protocol/Internet  Protocol  (UDP/IP)  is  used  for  communication  outside  of  the 
application.  The  BoltPMmount  code  in  the  game  engine  allows  access  through 
UDP/IP.  The  BOLT  code  allows  the  CARVE  GUI  to  communicate  through  to  the 
game  engine.  Commands  are  sent  from  the  GUI  to  the  vehicle,  directing  movements 
based  on  inputs  from  the  code,  keyboard,  joystick,  or  steering  wheel. 

The  platform  mobility  code  provides  access  to  the  vehicle  infonnation  while  the 
BOLT  sensor  provides  access  to  the  camera  buffers.  The  configuration  menu  on  the 
RIVET  startup  screen  allows  for  construction  of  vehicles,  as  well  as  sensors.  To 
use  a  vehicle  in  the  application,  a  vehicle  is  first  selected,  named,  and  then  sensors 
are  added.  A  separate  computer  running  RIVET  can  mount  a  BOLT  sensor  to 
connect  to  the  server  computer.  With  a  vehicle  properly  configured  with  BOLT,  the 
CARVE  application  is  able  to  connect  to  RIVET.  Directions  on  how  to  connect  to 
CARVE  are  provided  in  Appendix  K. 

4.3.1  Teleoperation,  Remote  Control,  Manual  Control 

CARVE  is  designed  to  work  with  a  number  of  controllers,  which  provides  increased 
flexibility  in  the  experimental  design.  Adapting  the  CARVE  program  code  to  set 
the  controllers  may  require  a  more  in-depth  understanding  of  computer 
programming.  However,  it  is  possible  to  customize  a  variety  of  controllers, 
including  the  following:  keyboard,  mouse,  touch  screen  GUI,  Logitech  Gamepad, 
Logitech  joystick,  Logitech  G27  wheel  and  pedals,  Microsoft  Kinect,  and  other  user 
interfaces  (e.g.,  engage/disengage  buttons). 

The  original  controllers  available  for  use  with  CARVE  were  the  keyboard  and 
mouse,  a  Logitech  gamepad,  and  a  touch  screen  track  ball.  The  keyboard  and  mouse 
controls  were  designed  to  mirror  the  user  interface  and  key  commands  described  in 
the  RIVET  Developer’s  Guide  (General  Dynamics  Robotic  Systems  2010,  48).  The 
Logitech  gamepad  controls  were  designed  to  provide  additional  fidelity  for 
interaction  with  a  vehicle.  In  addition,  each  button  is  programmable,  and  can  be 
used  for  dual-task  experimentation.  Figure  13  provides  a  visual  representation  for 
the  current  Logitech  setup. 
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Manual  Control  Mode 


Autonomous  Control  Mode 


Autonomous 

Control 

Mode 


Brake 


Speed  /  Steering  Target  ID 


Brake 


Look  Around  Target  ID 


SpeedUp 
+3  ninli 


Manual 

Control 

Mode 


SlowDown 

- 1  nipli 


Fig.  13  Logitech  gamepad  controller  current  button  layout 


The  touch  screen  track  ball  controls  for  speed  and  movement  are  directly  on  the 
CARVE  application  screen  (Fig.  14).  This  controller  interface  is  very  similar  to  a 
joystick.  The  user  will  move  the  circular  thumb  pad  to  drive  the  vehicle  in  the 
direction  the  center  ball  is  moved.  The  farther  from  center  the  ball  moves,  the  faster 
the  speed  at  which  the  vehicle  moves. 


Note:  The  red  text  and  outline  are  of  buttons  are  used  here  to  demonstrate  key  parts  of  the  GUI  for 
demonstration  purposes  only.  They  are  not  part  of  the  actual  GUI. 


Fig.  14  Touch  screen  track  ball  for  driving  controls 
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To  meet  the  diverse  needs  for  a  variety  of  experiments,  CARVE  has  been  updated 
with  the  capabilities  to  use  the  Microsoft  Kinect  sensor,  the  Logitech  G27  Racing 
Wheel  and  Pedals,  as  well  as  additional  engineered  controllers  (e.g.,  button 
activation  controls).  These  additional  user  inputs  allow  for  more  experimental 
control. 


The  Kinect  sensor  is  a  low-cost  (~$  150)  video  and  depth  sensor  with  an  open  source 
software  development  kit  that  allows  developers  to  use  the  device  for  gesture  input, 
as  well  as  spoken  language  (Fig.  15). 


Fig.  15  Microsoft  Kinect  sensor 

To  update  the  CARVE  for  manual  and  autonomous  self-driving  vehicle 
capabilities,  controls  for  the  Logitech  G27  Racing  Wheel  were  also  added  to  the 
CARVE  software.  The  current  functionality  allows  for  full  manual  control  driving. 
In  addition,  an  automation  button  system  was  built  and  programmed  to  engage  and 
disengage  the  vehicle’s  automation.  These  controllers,  when  used  together,  allow  a 
user  to  switch  between  autonomous  self-driving  vehicle  control  and  manual  control 
(Fig.  16). 
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Fig.  16  Logitech  G27  racing  wheel  and  pedals,  and  additional  button  controls 

4.3.2  Waypoint  Following 

CARVE  was  designed  to  integrate  waypoint  following  to  give  the  vehicle  a  level 
of  semi-autonomy  for  navigation  (Fig.  17).  Flere,  waypoints  are  markers, 
represented  by  an  orange  circle,  that  are  placed  in  set  locations  for  path  planning 
(navigation)  prior  to  the  start  of  a  mission.  Using  waypoint  following  allows  the 
user  to  conduct  additional  tasks  such  as  watching  a  video  stream  or  watching  a  radar 
screen  for  potential  enemies.  Waypoint  paths  are  created  on  the  map  screen  (Fig. 
17)  using  the  Add  Waypoint  button  [A],  Each  click  adds  an  additional  waypoint  to 
the  path.  The  entire  path  can  be  deleted  using  the  Delete  button  [B].  Once  a  path  is 
complete  the  Change  button  [C]  is  used  to  adjust  waypoints  on  the  screen. 
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4.3.3  Autonomous  Capabilities 


While  the  original  waypoint-following  capabilities  mirrored  real-world  semi- 
autonomous  navigation  through  the  placement  of  different  waypoints,  it  was 
important  to  advance  the  capabilities  of  the  system  to  include  a  customizable 
control  system.  For  this  purpose,  dynamic  capabilities  were  added  to  the  waypoints 
(Fig.  18).  Step-by-step  directions  for  setting  dynamic  waypoints  are  included  in 
Appendix  L. 


Fig.  18  Options  for  dynamic  waypoints  in  CARVE 
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A  feature  that  was  added  to  the  latest  release  of  CARVE  was  to  allow  adjustment 
of  actions  between  waypoints.  Upon  adding  a  waypoint  to  the  map,  the  Waypoint 
Action  dialog  box  opens  and  an  action  can  be  selected  from  the  list  (Fig.  19  and 
Fig.  20).  If  “None”  is  selected,  then  the  same  specifications  from  the  previous 
waypoint  are  kept.  This  option  can  be  used  for  “navigation  only”  markers.  If  “Stop” 
is  selected,  then  the  vehicle  will  stop  at  that  waypoint  for  the  “duration”  set  in  the 
last  text  box.  Duration  time  is  set  in  milliseconds.  If  “Estop”  is  selected,  the  vehicle 
will  remain  stopped  until  the  user  releases  the  vehicle  by  re-engaging  the 
automation  through  a  button  press.  This  capability  allows  the  experimenter  to  set 
breaks  in  the  simulation.  The  “SetSpeed”  selection  is  coupled  with  the  “Speed” 
entered.  The  speed  is  set  in  miles  per  hour  (mph).  Finally,  the  “Destination”  option 
functions  similar  to  the  “None”  in  that  it  provides  a  navigational  marker.  However, 
it  is  has  a  different  link  to  an  output  file  that  differentiates  between  these  points. 
Therefore,  it  can  be  used  to  record  time  stamps  for  specific  events. 


Fig.  19  Waypoint  Action  menu 
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Fig.  20  Dynamic  waypoints  available  actions 

4.4  Alternative  Tasks 

Additional  tasks  that  participants  can  complete  can  also  be  added  to  the  CARVE 
interface.  These  customizable  tasks  can  be  used  for  a  dual-task  paradigm  or  to 
increase  the  participant’s  workload.  One  such  example  is  the  radar  task  (Fig.  21). 
For  this  task,  participants  will  monitor  the  radar  sensor  and  identify  when  a  blip 
(small  colored  dot)  changes  from  yellow  (target)  to  red  (threat)  by  pressing  a  button 
on  the  user  control  interface.  Once  the  threat  is  identified,  the  blip  will  turn  from 
red  to  blue  providing  feedback  to  the  participant  that  their  task  was  recorded. 
CARVE  is  set  up  to  record  perfonnance  on  this  task  to  a  .csv  file.  Performance 
metrics  include  correct  detection,  false  alarms,  misses,  and  detection  time.  This  task 
is  also  customizable  to  create  conditions  of  varying  workload  (e.g.,  number  of 
possible  targets,  time  between  targets,  speed  of  movement,  or  pattern  of 
movement).  Instructions  are  provided  in  Appendix  M. 
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Fig.  21  Alternative  radar  target  detection  task  in  CARVE 

4.5  Manipulating  and  Measuring  HRI 

RIVET  and  CARVE  provide  a  number  of  customizable  capabilities  for 
manipulating  and  measuring  HRI.  This  section  provides  a  brief  review  of  some  of 
those  capabilities  for  data  collection  as  well  as  references  for  others.  These  may 
include  recording  the  behaviors  of  the  simulated  robot  or  virtual  Soldier  avatar  to 
determine  frequency  and  duration  of  HRI,  or  data  for  use  in  an  after-action  review 
(AAR).  An  AAR  is  a  review  of  the  simulation  experiment  after  the  fact,  where 
participants  can  provide  opinions  and  feedback  on  their  personal  assessment  of  the 
experiment.  Such  AAR  data  can  supplement  quantitative  data  collected  from  the 
simulation  and  interaction.  Depending  on  what  data  from  the  simulation  or  the 
interface  are  needed,  the  code  base  can  be  developed  to  gather  and  export  that 
information  to  an  output  file  through  RIVET  or  CARVE. 

RIVET  has  a  capability  of  recording  data  from  the  game  engine  and  the  scene.  This 
is  done  using  code  in  the  engine  to  allow  for  Structured  Query  Language  (SQL) 
calls  to  capture,  record,  and  manage  data  held  in  a  relational  database.  For  more 
information  on  how  to  read,  write,  and  append  custom  files,  see  FilelO  Tutorial 
(http://docs.garagegames.com/tgb/official/)  or  the  section  of  the  Game 
Programmer’s  Guide  to  Torque  on  writing  and  appending  files  Section  9.5.9 
Writing  and  Appending  Files  of  the  Game  Developer’s  Guide  to  Programming 
Torque  (Maurina  2006,  370). 

There  are  also  ways  of  recording  data  using  the  CARVE  interface.  A  class  was 
created  named  “WWDataLogger”,  which  was  used  to  gather  data  from  user 
interaction  with  the  experiment.  In  this  example,  all  of  the  data  is  gathered  and 
written  to  a  Microsoft  Excel  file.  It  records  the  location  of  the  target  blip,  the 
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activity  (path  started,  enemy  in  zone,  enemy  identified,  enemy  misidentified,  and 
path  completed),  and  a  time  stamp.  From  this,  it  is  possible  to  calculate  signal 
detections,  which  are  delineated  as  correct  response,  errors  (misses, 
misidentifications),  and  detection  time.  Data  collection  can  also  be  customized  to 
record  robot  behaviors  and  number  of  interventions  in  the  robot’s  autonomy  with 
recording  of  an  event,  time,  and  associated  data  (e.g.,  duration  and  associated  times 
of  button  presses,  type  of  button,  as  well  as  time  and  frequency  between  manual 
and  autonomous  modes).  This  type  of  data  collection  requires  more  advanced 
programming  to  create  the  output  file  (.csv  or  Microsoft  Excel).  Depending  on  what 
data  from  the  simulation  or  the  interface  are  needed,  the  code  base  can  be  developed 
to  gather  and  export  that  infonnation  into  a  separate  file. 

Customizations  to  the  simulation  environment  can  be  made  in  line  with  the  HRI 
metrics  by  Steinfeld  et  al.  (2006),  Goodrich  and  Schultz  (2007),  and  Weiss  et  al. 
(2009),  as  previously  described  in  Section  2. 1  of  this  report. 

•  Navigation  of  the  simulated  robot:  The  effectiveness  (e.g.,  number  of 
obstacles  avoided  or  tasks  completed)  and  efficiency  (e.g.,  time  and 
duration)  related  to  navigation  are  elements  that  can  be  recorded  in  RIVET 
(.sql),  CARVE  (.csv),  or  by  hand  during  an  experiment  (e.g.,  pen  and 
paper).  Participant  feedback  through  an  AAR  or  video  analysis  can  be 
accomplished  by  recording  the  mission  run  for  later  viewing  through  the 
integration  of  additional  video  recording  software,  such  as  Fraps 
(www.fraps.com). 

•  Perception  from  sensor  data:  RIVET  provides  simulated  sensor  readings 
that  come  in  the  same  format  that  a  user  would  receive  on  the  real  robot. 
Sensor  data  can  then  be  run  through  perception  algorithms  to  identify  how 
well  the  robot  understands  the  world  around  it.  Current  configurations  of 
the  simulated  robot  can  include  camera,  lidar,  and  semantic  sensors 
(Gonzalez  et  al.  2009).  The  details  for  collecting  and  assessing  sensor 
perception  data  are  outside  the  scope  of  this  report.  However,  for  more 
infonnation,  Dean  and  DiBerardino  (2014)  provide  resource  material  on  the 
integration  of  the  Common  World  Model. 

•  Management  and  function  allocation:  Since  this  metric  area  is  related  to 
user  interfaces  and  control,  we  recommend  using  customizable  data 
recording  through  the  CARVE  application.  It  is  possible  to  set  the 
frequency,  duration,  type,  degree  of  movement,  etc.  for  every  intervention 
using  a  user  control  interface  to  record  into  a  .csv  output  file.  However,  this 
may  require  more  advance  programming  capabilities. 
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•  Manipulation :  Here,  manipulation  is  referring  to  how  the  simulated  robot  is 
directly  influencing  the  VE.  RIVET  offers  some  degree  of  manipulation 
required  for  HRI  type  tasks;  however,  fine-grained  manipulation  required 
by  a  real-world  robot  is  nearly  impossible  to  accurately  model  in  this 
simulation  environment.  This  is  because  RIVET  is  built  on  a  game  engine 
instead  of  in  a  high-performance  computing  simulation.  However,  assessing 
HRI  may  not  require  fine-grained  manipulation.  Therefore,  we  recommend 
that  experimenters  refer  to  the  design  guidelines  and  the  requirements 
documents  of  the  real-world  robot.  Programmable  animation  sequences  can 
then  be  built  and  integrated  into  the  VE  (General  Dynamics  Robotic 
Systems  2010,  67-232). 

•  Social  interaction :  There  are  a  number  of  opportunities  for  manipulating 
and  recording  social  interaction  within  RIVET  and  CARVE,  as  well  as  with 
obtaining  feedback  after  the  fact  with  an  AAR.  These  measures  include,  but 
are  not  limited  to,  recorded  distances  between  people  and  robots, 
manipulation  of  behaviors,  communication  exchanged  (e.g.,  programmable 
feedback  system),  programmable  team  roles,  as  well  as  usability,  social 
acceptance,  and  individual  differences.  These  metrics  can  be  recorded 
through  the  development  of  customizable  GUIs,  programmable  pauses, 
recordable  behaviors,  and  even  integration  of  outside  tools  (e.g.,  eye 
tracking). 

5.  Previous  HRI  Research  Using  RIVET 

A  number  of  research  efforts  through  the  RCTA  have  used  various  functionalities 
of  RIVET  and  CARVE  for  HRI  experimentation.  One  of  the  main  goals  of  this  type 
of  experimentation  was  the  advancement  of  robot  design  through  the  consideration 
of  the  human  or  the  human-robot  team.  As  mentioned  earlier  in  this  report,  the 
research  discussed  here  is  in  line  with  ARL’s  Intelligent  Systems  Enterprise  vision 
that  enables  the  teaming  of  autonomous,  intelligent  systems  with  Soldiers.  This 
section  discusses  experiments  in  both  dynamic  and  unstructured  combat 
environments,  as  well  as  noncombat  military  installations  and  base  operations.  For 
additional  resources  on  similarities  and  differences  between  RIVET  and  field 
experiments  in  general,  see  Bodt  et  al.  (2010)  and  Schafer  et  al.  (2015).  This  section 
demonstrates  the  diversity  and  customizability  of  RIVET,  which  make  it  a  valuable 
platform  for  HRI  experimentation. 
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5.1  Cordon  and  Search 


The  cordon  and  search  task  has  been  used  as  a  motivating  scenario  by  ARL 
researchers.  During  urban  transit  by  a  small  (4-5  Soldier)  unit,  a  fugitive  is  reported 
to  have  entered  a  building  that  the  unit  is  approaching.  A  human-transportable  robot 
is  instructed  to  “screen  the  back  door”  of  the  building  by  the  unit  commander  since 
he  cannot  safely  split  up  his  limited  resources  (Fig.  22).  While  this  narrative  occurs 
in  the  context  of  a  cordon  and  search  operation,  its  underlying  capabilities  support 
a  broad  range  of  potential  operational  missions. 


1.  Mission  order:  “Watch  the  back  of  that 
building  and  report  suspicious  activity” 


2.  Understand  command,  relate  to  perceived 
environment  (“that  building”),  and 
demonstrate  understanding  (e.g.,  lase  building) 


3.  Figure  out  where  to  go  to  surveil  building, 
plan/execute  path  to  move  safely  &  securely, 
establish  initial  shared  SA  and  Common  Ground 


4.  Move  through  cluttered  environment  to 
reach  OP,  overcome  mobility  challenges 


5.  Approach  OP  cautiously  —  along  wall  of 
building 


7.  Assess  egress  points,  detect  humans,  update 
status 


6.  As  needed,  clear  obstacles  and  get  in 
advantageous  position  for  surveillance 


8.  Upon  mission  completion,  rejoin  unit, 
maintain  SA,  accept  new  missions 


Note:  OP  =  objective  point. 


Fig.  22  RCTA  motivating  scenario  (ARL  Robotics  Collaborative  Technology  Alliance  2012) 
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5.1.1  Target  Detection:  Person 

In  this  experiment,  participants  monitored  a  Talon  robot  complete  the  cordon  and 
search  type  task  to  monitor  the  back  of  a  building  for  human  terrorist  targets 
(Schaefer  2013,  2016).  A  single  computer  (RIVET  only)  setup  was  ideal  since 
participants  viewed  the  Talon  robot’s  camera  sensor  from  a  remote  location  and 
were  unable  to  directly  intervene  in  the  robot’s  behavior.  For  the  purpose  of  this 
experiment,  a  single  video  of  the  simulation  was  created  from  the  camera-view  on 
the  Talon  robot  that  showed  the  robot  navigating  to  the  back  of  a  building,  finding 
a  secure  location,  and  monitoring  the  back  door  for  human  targets.  The  video  of  the 
simulation  was  created  using  Fraps  real-time  video  capture  and  benchmarking 
program  with  a  30  frame/second  (fps)  .avi  file.  The  .avi  file  was  converted  into  the 
.mp4  file  fonnat  to  add  auditory  feedback  from  the  Talon  robot  in  post-processing. 
This  allowed  the  experimenter  to  create  multiple  conditions  from  a  single 
simulation.  Condition  1  provided  a  100%  reliable  detection  condition  in  which  the 
Talon  robot  provided  verbal  feedback  (i.e.,  “target  detected”)  for  all  the  human 
targets.  Condition  2  provided  a  25%  reliable  detection  condition.  While  auditory 
feedback  could  have  been  added  within  RIVET,  it  would  have  required  the  creation 
of  separate  mission  files.  Thus,  use  of  outside  software  was  ideal. 

During  development,  the  VE  used  was  a  previously  developed  base  environment 
named  CACTF  (Combined  Anns  Collective  Training  Facility).  This  VE  was 
originally  developed  to  represent  a  simulated  version  of  a  real-world  training  facility 
of  an  urban  environment  that  was  designed  to  conduct  multi-echelon,  full-spectrum 
operations  training  up  to  battalion  task  force  level.  The  base  VE  included  the  layout 
of  the  physical  environment  (e.g.,  ground,  roadways,  buildings,  and  lighting).  To 
meet  the  needs  of  this  mission,  terrorist  NPCs  were  created  and  added  to  RIVET 
(Fig.  23).  More  infonnation  on  creating  NPCs,  animations,  and  adding  to  the  TGE 
can  be  found  in  the  RIVET  Developer ’s  Guide  (General  Dynamics  Robotic  Systems 
2010). 
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Fig.  23  Terrorist  NPCs 

Additional  capabilities  were  added  to  RIVET  to  meet  the  needs  of  this  experiment. 
Task-specific  customization  of  the  environment  was  accomplished  through  creation 
of  RIVET  GUIs,  as  well  as  Scripting  syntax  using  Torsion.  Torsion  is  a  powerful 
integrated  development  environment  for  creating  TorqueScript-based  games  and 
modifications.  Specific  customization  included  entering  objects,  obstacles,  and 
creation  of  paths,  to  name  a  few.  For  customizable  control  of  the  Talon,  additional 
script  files  were  included  to  create  and  set  up  a  path-following  Talon  robot  (see 
Appendix  F  and  Appendix  G  for  direction  on  adding  a  path- following  vehicle).  This 
allowed  a  preprogrammed  path  and  the  robot  behaviors  to  be  set  before  recording 
the  mission.  This  addition  to  RIVET  has  resulted  in  a  number  of  vehicles  (e.g., 
Humvee,  taxi,  KBot,  Talon)  with  path-following  capabilities  that  are  available  for 
future  simulation  experiments. 

5.1.2  Target  Detection:  Object 

This  experiment  used  a  single-user  fonnat  (one  RIVET  computer)  and  was 
designed  without  adding  any  new  technology  or  requiring  any  additional 
TorqueScript  files.  For  this  experiment,  participants  monitored  a  computer  agent 
(either  a  Soldier  avatar  or  a  Talon  robot)  as  they  located  16  missing  M16  rifles 
throughout  a  Middle  Eastern  town  (Sanders  et  al.  2012).  Two  videos  of  the 
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simulation  (one  for  a  100%  reliable  agent  and  one  for  a  25%  reliable  agent)  were 
created  from  the  third  person  perspective  for  each  agent  navigating  through  the 
environment  and  locating  targets. 

The  VE  used  a  base  environment  of  the  Middle  Eastern  Marketplace  that  already 
had  static  and  dynamic  objects,  and  a  crowd  of  people.  To  meet  the  needs  of  this 
mission,  a  path-following  Soldier  and  a  path-following  Talon  robot,  as  well  as  the 
missing  rifles  were  added  to  the  environment  (Fig.  24).  This  allowed  additional 
control  in  navigation  path  and  timing  between  the  agents.  The  video  of  the 
simulation  was  created  using  Fraps  real-time  video  capture  and  benchmarking 
program  with  a  30  fps  .avi  file.  For  the  purposes  of  this  work,  an  external 
physiological  measurement  tool  (Labscribe;  iWorx,  Inc.)  and  an  eye  tracker 
(EyeWorks;  Eyetracking,  Inc.)  were  used  to  collect  physiological  and  behavioral 
data.  Results  are  provided  in  Sanders  et  al.  (2012). 


Fig.  24  Example  of  RIVET  map,  targets,  and  avatars  (Talon  and  Soldier) 

5.1.3  Navigation 

For  this  experiment,  participants  controlled  a  Soldier  agent  situated  within  a  Middle 
Eastern  environment  (Mid  East  RCTA  Demo).  The  task  was  to  assist  an 
autonomous  Talon  robot  from  a  set  location  to  a  rendezvous  point  as  quickly  as 
possible  (Schaefer  2013,  2015).  This  assistance  required  the  participant  to  locate 
the  robot  and  move  obstacles  out  of  the  path  if  the  robot  was  unable  to  navigate 
around  these  obstacles.  Participants  completed  this  task  with  2  different  robots.  The 
first  robot  was  80%  reliable  in  self-navigating  obstacles  (successfully  navigated 
around  4  out  of  the  5  potential  obstacles).  The  second  robot  was  only  20%  reliable 
(successfully  navigated  around  one  obstacle). 

Due  to  the  design  of  this  experiment,  moveable  objects  were  needed  and  created  by 
General  Dynamics  Robotic  Systems  (GDRS)  for  use  in  RIVET.  This  included 
syntax  for  a  refrigerator,  box,  crate,  barrel,  and  trashcan  (refer  to 
C:\RIVET\sim.base\server\moveableobjects.cs).  In  addition,  independent  task- 
specific  customization  of  the  physical  environment  was  accomplished  through 
TorqueScript  syntax.  Specific  customization  included  entering  objects,  obstacles, 
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creation  of  paths,  etc.  Scripting  files  were  created  for  both  the  training  session  and 
the  task  conditions.  The  simulated  tasks  were  recorded  using  Fraps  real-time  video 
capture  and  benchmarking  program  with  a  30-fps  .avi  file.  Video  was  recorded 
from  the  Soldier  character’s  perspective  for  later  data  analysis. 

5.1.4  Transparency 

Transparent  bi-direction  communication  is  an  extremely  important  component  to 
effective  and  trusted  human-robot  teaming  (Barnes  et  al.  2016;  Schaefer  et  al. 
2016).  To  address  this  HRI  issue,  Sanders  et  al.  (2014)  used  RIVET  as  a  single 
computer  architecture  for  a  joint  Soldier-robot  team  responsible  for  clearing  an  area 
of  weapons  and  locating  civilians  by  marking  their  location  on  a  map.  The  authors 
did  not  require  any  additional  functionality  from  RIVET  for  this  task.  They  did, 
however,  integrate  the  RIVET  simulation  with  E-Prime  to  adapt  the  type  of 
communication  from  the  robot.  This  included  variations  in  the  mode  of 
communication  from  the  robot  (e.g.,  audio,  text,  graphics)  and  the  amount  of 
information  being  communicated  (e.g.,  minimal,  contextual,  constant). 

5.2  Gunnery 

A  second  type  of  combat-related  mission  was  created  in  RIVET  to  conduct  virtual 
unmanned  gunnery  exercises.  The  purpose  was  to  assess  the  effect  of  various 
interface  devices  for  robotic  asset  control  on  system  operation  and  overall 
performance.  A  series  of  gunnery  exercises  was  created  to  assess  the  ability  of  the 
operator  to  identify  targets,  select  the  appropriate  weapon,  and  engage  targets  with 
the  handheld  gamepad  and  Microsoft  Kinect  voice  commands.  This  experimental 
setup  required  use  of  the  client/server  architecture  with  the  RIVET  server,  BOLT 
client,  and  CARVE  application.  The  VE  in  RIVET  adapted  the  base  Engineering 
Test  World  mission  to  meet  the  needs  of  this  type  of  task,  including  the  terrain, 
foliage,  roadways,  and  targets  (Fig.  25). 
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Configuration: 
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Fig.  25  Gunnery  range  map 

These  exercises  required  additional  components  to  be  set  up  within  RIVET, 
including  a  number  of  targets,  type  of  targets,  the  posture  of  the  vehicle  (offensive 
or  defensive),  and  the  position  on  the  range  where  it  will  take  place.  These  data  are 
entered  into  the  system  using  a  custom  GUI  (Fig.  26).  When  using  this  GUI,  the 
data  entered  are  added  to  the  database  using  SQL  calls  in  the  script.  Once  the 
database  is  updated  with  the  exercises,  the  experiment  can  be  conducted.  The 
exercise  begins  and  the  timings  of  all  events  that  happen  during  that  engagement 
are  recorded  in  the  database  for  use  during  an  AAR. 
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Note:  The  GUI  creation  editor  gives  the  programmer  all  the  tools  necessary  to  create  these  and  other  interfaces. 
New  controls  are  found  in  the  upper  left  drop-down  box  (outlined  in  red),  and  once  they  are  added  to  the  canvas, 
they  show  up  in  the  GUI  tree  view  (outlined  in  green)  with  their  properties  accessible  in  the  inspector  dialog 
(outlined  in  blue). 

Fig.  26  GUI  editor:  example  of  gunnery  user  interface  GUI  setup 

5.3  Driverless  Vehicle  Transport 

A  third,  and  most  recent,  type  of  mission  used  for  HRI  experimentation  investigated 
a  specific  non-combat  operation  of  driverless  vehicles  for  passenger  transit.  One 
example  is  an  experiment  motivated  by  the  Autonomous  Robotics  for  Installations 
and  Base  Operations  (ARIBO)  project  (Marshall  2014).  The  purpose  of  this 
experiment  was  to  build  a  virtual  US  Army  post  in  RIVET  in  order  to  assess 
behavior  and  performance  of  a  person  on  board  a  self-driving  vehicle  prior  to  the 
actual  vehicle  being  developed  (Schaefer  and  Scribner  2015).  This  design  required 
use  of  the  client/server  architecture  with  the  RIVET  server,  BOLT  client,  and 
CARVE  application. 

The  first  major  need  was  to  build  the  VE  within  RIVET  (refer  to  Appendixes  A-I). 
For  this  design,  the  ARL  Test  World  was  used  as  the  base  virtual  terrain.  The  height 
of  the  terrain  varies  as  seen  from  the  mountain  ranges  in  the  background  to  the 
mostly  level  terrain  in  the  cantonments  area.  However,  this  open  expanse  amidst  a 
mountain  range  was  sufficient  for  this  mission  environment.  The  ground  has 
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multiple  textures  including  grass,  dirt,  and  hard-packed  black  gravel.  These  were 
added  to  the  scene  with  the  Terrain  Texture  editor  that  allows  the  creator  to  add 
color  and  dimension  to  the  scene.  Then,  the  medical  facilities  from  a  US  military 
installation  were  used  as  a  frame  of  reference  for  developing  a  road  and  sidewalk 
structure,  building  location,  parking  lots,  and  other  stationary  obstacles  (Fig.  27). 
Many  of  the  static  objects  included  in  this  new  mission  were  already  available 
within  the  RIVET  libraries.  While  this  means  that  the  VE  is  not  an  exact  replica  of 
the  real  environment,  it  does  meet  the  requirements  for  the  experiment  and  provides 
the  capability  to  include  some  of  the  key  requirements  (e.g.,  vehicle  behaviors, 
transportation  times,  number  of  turns)  to  be  able  to  transition  this  experimental 
design  to  the  real  world  once  the  prototype  robotic  vehicle  is  available  for  further 
experimentation. 


Fig.  27  Virtual  environment  representing  the  medical  facilities  on  a  US  Army  post 

In  the  next  step  after  building  the  VE,  the  CARVE  application  was  adapted  in 
multiple  ways.  First,  dynamic  waypoints  were  included  (refer  to  Section  4.2.3)  to 
more  accurately  mirror  real-world  autonomous  behaviors.  Second,  a  new 
controller,  the  Logitech  G27  wheel  and  pedals,  as  well  as  the  engage/disengage 
buttons,  were  integrated  into  CARVE  for  realistic  manual  control.  Third,  an  output 
file  (.csv)  was  created  that  recorded  all  interventions  in  the  vehicle’s  autonomy. 
Finally,  a  number  of  GUIs  were  developed  within  RIVET  to  provide  instructions 
and  information  to  the  participant  (e.g.,  notice  of  other  passengers  onboard  the 
vehicle,  location  infonnation,  and  wrong-way  directions). 

The  final  step  was  to  choose  a  robotic  vehicle  that  was  approximately  the  same  size 
and  shape  of  the  real-world  prototype  vehicle.  Even  though  there  are  many  different 
robotic  vehicles  to  choose  from  within  the  simulation,  it  is  important  to  match  the 
system  to  the  exercise.  A  larger  vehicle  was  needed  for  this  experiment,  as  it  has  to 
be  able  to  pick  up  and  discharge  passengers.  Since  details  were  not  available  about 
the  final  physical  design  of  the  real-world  prototype  vehicle,  the  tactical 
autonomous  combat  chassis  (TAC-C)  was  used,  which  was  previously  designed  by 
GDRS  as  a  simulated  manned/unmanned  platform  with  the  potential  for  manual  or 
autonomous  control  (Fig.  28).  In  addition,  the  camera  position  was  moved  to  be 
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located  at  the  “driver”  position  of  the  vehicle,  so  very  little  of  the  vehicle  was 
actually  seen  by  the  participant.  Additional  information  on  this  set  of  experiments 
is  currently  available  in  Schaefer  and  Scribner  (2015)  and  will  be  available  in 
additional  ARL  technical  reports. 


Note:  The  TAC-C  was  used  because  of  its  size  (height  and  width),  as  well  as  the  functionality  of 
the  animation  sequences.  Originally,  the  vehicle  had  a  turret  and  other  associated  weaponry  (left). 
Once  the  graphics  of  the  turret  and  other  weaponry  were  removed,  the  functionality  to  scan  right 
to  left  as  well  as  up  and  down  was  still  there  (right).  We  used  this  functionality  to  represent 
someone  turning  their  head  to  look  around  for  oncoming  traffic  or  environmental  obstacles. 

Fig.  28  Customizations  to  the  TAC-C 


6.  Discussion 


This  report  provides  an  introduction  to  developing  and  advancing  RIVET 
simulation  and  the  associated  CARVE  application  to  meet  the  needs  for  HRI 
experimentation.  Overall,  the  capability  for  customization  from  the  user  control 
interface  to  the  sensor  and  autonomy  integration  and  on  to  the  integration  of  a  user 
feedback  interface  make  the  RIVET  simulation  system  a  valuable  resource  to 
understanding  the  needs  of  a  variety  of  current  and  future  HRI  (Section  5).  Further, 
step-by-step  instructions  are  provided  for  future  researchers  to  set  up  and  use 
RIVET  (Appendixes  A-M).  This  section  provides  some  remaining  lessons  learned 
while  updating  the  RIVET  system,  as  well  as  additional  resource  guides. 

6.1  Lessons  Learned 

This  section  addresses  some  of  the  solutions  to  problem  areas  found  while 
developing  and  advancing  RIVET  and  the  associated  CARVE  application. 


Approved  for  public  release;  distribution  is  unlimited. 


38 


6.1.1  Problem  No.  1:  CARVE  Does  Not  Always  Load  Correctly 

If  CARVE  does  not  load,  gives  an  error,  or  shuts  down,  then  first  check  the 
connectivity  to  make  sure  the  CARVE  computer  is  receiving  signals  from  both  the 
RIVET  server  and  the  BOLT  client  computers  (Appendix  B,  Figs.  B-l  and  B-2).  It 
is  good  practice  to  look  at  the  connectivity  prior  to  loading  CARVE  to  check  that 
CARVE  is  set  up  to  connect  to  the  RIVET  and  BOLT  IP  addresses  through  the 
Ethernet  LAN. 

The  second  check  is  the  RIVET  server  machine.  When  using  the  3-computer  setup 
(RIVET,  BOLT,  CARVE),  it  is  vital  to  choose  a  vehicle  that  has  a  BOLT  sensor 
attachment.  Otherwise,  the  BOLT  sensor  on  the  client  machine  and  the  CARVE 
application  will  not  connect.  If  the  correct  vehicle  was  chosen,  then  check  the 
BOLT  client  machine.  Double-check  to  see  that  the  name  is  set  to  User  1 ,  the  name 
of  the  vehicle.  This  name  is  what  allows  the  BOLT  sensor  to  attach  to  the  vehicle. 

If  there  are  still  issues  connecting  to  the  CARVE  application,  the  final  thing  to 
check  is  the  map.  CARVE  requires  the  map  of  the  VE  from  RIVET  to  function 
appropriately.  The  optimal  sizing  for  the  map  is  1600  x  1600  pixels  and  less  than  2 
MB  in  size.  External  programs  can  be  used  to  reduce  the  size  of  the  .png  file  (e.g., 
Adobe  Fireworks).  Directions  for  creating  a  map  are  provided  in  Appendix  B. 

6.1.2  Problem  No.  2:  CARVE  Takes  Too  Long  to  Load 

CARVE  can  take  upwards  of  a  minute  to  initially  load  the  application,  because  it  is 
pulling  the  mission  map  from  RIVET  every  time  it  is  initialized.  When  conducting 
multiple  trials,  this  time  delay  can  add  up  to  a  great  deal  of  wait  time.  However,  to 
address  this  issue,  we  built  in  a  shortcut  that  will  store  the  map  locally  following 
the  first  initialization  of  CARVE.  Directions  are  included  in  Appendix  K. 

6.1.3  Problem  No.  3:  Picture  Quality  Is  Granulated  or  Pixelated 

If  the  picture  becomes  granulated  or  pixelated,  this  may  be  a  sign  of  a  graphics  card 
issue.  Building  complex  VEs,  especially  those  with  a  number  of  moving  objects, 
can  slow  down  the  system.  One  solution  is  to  reduce  the  number  of  static  and 
dynamic  objects  in  the  VE.  However,  that  may  not  always  be  possible.  A  second 
solution  is  to  use  trigger  points  to  create  and  delete  NPC  and  path-following 
vehicles  at  set  points  in  time  and  space  during  the  mission  run  (Appendixes  E-H). 
Finally,  check  the  hardware — more  specifically,  the  graphics  card.  Nvidia  GeForce 
graphics  cards  were  found  to  be  better  suited  to  work  with  complex  VEs  within 
RIVET  than  the  Nvidia  Quadro  cards.  In  addition,  it  may  be  beneficial  to  have  a 
graphics  card  with  a  separate  memory  storage  (e.g.,  Nvidia  GeForce  GTX  780). 
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If  this  issue  only  occurs  on  the  CARVE  display,  then  it  may  be  a  packaging  issue. 
CARVE  pulls  the  camera  video  from  the  BOLT  machine  by  decompressing  the 
video  stream  and  then  enlarging  it  on  the  CARVE  screen.  This  process  of  shrinking 
and  then  enlarging  the  video  stream  can  lead  to  pixelation.  One  possible  solution  is 
to  reduce  the  screen  size  on  the  BOLT  machine  by  clicking  on  the  Configuration 
button  prior  to  mounting  the  BOLT  sensor  (Fig.  29). 


Fig.  29  Location  to  change  the  screen  size 


6.1.4  Problem  No.  4:  Camera  Sensor  Is  Located  in  the  Wrong  Place 

It  is  possible  that  the  standard  camera  sensor  location  is  not  in  ideally  placed  to 
meet  the  needs  of  the  experiment.  For  example,  if  the  camera  is  placed  too  high  or 
too  low,  a  Soldier  avatar  may  appear  to  be  taller  or  shorter  than  an  average  person 
appears.  Similarly,  the  height  of  the  camera  sensor  on  a  vehicle  may  give  the 
perspective  of  driving  on  roof  if  placed  too  high,  or  display  the  animations  of  the 
wheel  wells  if  placed  too  low.  In  addition,  the  forward  or  rear  placement  of  the 
camera  sensor  can  change  the  perspective  of  the  experiment.  For  example,  with  a 
Soldier  avatar,  it  is  possible  to  change  the  camera  placement  to  be  in  front  of  the 
Soldier,  at  eye  level  of  the  Soldier,  or  behind  the  Soldier. 
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The  solution  for  changing  the  placement  of  the  camera  is  simple,  but  it  requires  an 
understanding  of  the  TorqueScript  files.  Every  player  vehicle  has  an  associated 
TorqueScript  file  (.cs)  that  can  be  located  in  the  following  directory: 
C:\RIVET\sim.base\server.  To  edit  the  specific  vehicle  file  (.cs),  it  will  need  to  be 
opened  as  a  text  file  in  Notepad,  Notepad++,  or  Torsion. 

The  following  are  step-by-step  directions  on  how  to  adapt  the  camera  sensor 
placement  for  the  TAC-C. 

1 .  Open  the  tacc_weapons.cs  file  (C:\RIVET\sim.base\server). 

2.  Scroll  down  or  search  (Ctrl+f)  for  //3rd  person  camera  settings  (around  line 
98). 

3.  To  change  the  camera’s  distance  either  forward  or  backward  from  the 
middle  of  the  vehicle,  change  the  cameraMinDist  number.  The  max 
numbers  are  +8.0  and  -8.0. 


4.  To  change  the  vertical  distance  higher  or  lower,  change  the  cameraOffset 
with  a  positive  or  negative  number  (Fig.  30). 


tacc_  weapons.es 

(Globals) 

▼  serverCmdAddT  accWeapon 

97  I 

98  //  3rd  person  camera  settings 

99  cameraRoll  ■  true;  //  Roll  the  camera  with  the  vehicle 

100  cameraMinDist  *5.8;  //  Far  distance  from  vehicle 

101  //  original  cameraOffset  «  1.5;  //  Vertical  offset  from  camera  mount  point 

102  cameraOffset  *  -0.25;  //  Vertical  offset  from  camera  mount  point 

103  cameraLag  -  0;  //  Velocity  lag  of  camera 

104  cameraDecay  =  1;  //  Decay  per  sec.  rate  of  velocity  lag 

105  |  / 

Note:  Changing  the  location  of  the  camera  sensor  in  this  file  will  change  the  placement  of  the  camera  sensor 
for  this  vehicle  when  used  in  any  mission  in  the  future.  Therefore,  it  may  be  beneficial  to  add  a  comment  (green 
text  denoted  with  2  forward  slashed  lines)  marking  changes  made  to  the  TorqueScript  file. 

Fig.  30  TorqueScript  file  in  Torsion  for  adapting  the  camera  sensor  placement  for  the 
TAC-C  vehicle 

6.1.5  Problem  No.  5:  Need  to  Customize  Pre-loaded  Animation 
Sequences 

The  current  version  of  RIVET  is  designed  to  initialize  some  set  animation 
sequences  upon  inclusion  in  a  mission  environment.  For  example,  in  the  experiment 
on  target  detection  (Section  5.1 .2),  upon  placing  the  M16  rifles  within  the  VE,  they 
began  to  spin,  as  is  common  in  game  design.  This  animation  sequence  may  or  may 
not  be  ideal  for  a  specific  experiment;  however,  this  animation  sequence  has  also 
been  programmed  to  be  customizable.  In  other  words,  it  is  possible  to  stop  the  rifles 
from  spinning  in  circles.  Although  this  specific  example  refers  to  rifles  that  spin, 
other  animation  sequences  might  be  customized  in  a  similar  manner.  Some  brief 
directions  to  customize  are  as  follows: 

Approved  for  public  release;  distribution  is  unlimited. 


41 


1 .  In  RIVET,  press  F 1 1  to  open  the  World  Editor. 

2.  Press  F4  (or  use  the  Window  menu)  to  open  the  World  Editor  Creator. 
Select  and  add  the  M16  rifle  from  the  menu  on  the  right  side  of  the  screen. 

3.  To  add  the  M16  rifle,  select  “shapes”,  “weapons”,  and  “Ml 6”.  A  spinning 
rifle  should  be  visible  in  the  VE. 

4.  Click  on  the  Ml 6.  Once  it  has  a  yellow  box,  then  press  F3  (or  use  the 
Windows  menu)  to  open  the  World  Editor  Inspector  to  customize  the  M16 
rifle. 

5.  To  stop  the  rifle  from  spinning,  deselect  the  Rotate  button  and  click  apply 
(Fig.  31). 

6.  The  rifle  will  continue  spinning.  Go  to  File,  click  Save,  and  then  logout  of 
RIVET  completely.  After  logging  back  in  to  the  Mission  file,  the  rifle 
should  no  longer  be  spinning. 

7.  Another  way  to  ensure  that  it  accepted  the  changes  is  by  looking  at  the 
TorqueScript  file.  To  do  this,  open  the  Mission  File  (.mis)  by  going  to  the 
directory  C:\RIVET\sim.base\data\usermissions\  and  select  the  correct 
Mission  file.  At  the  bottom  of  the  file,  the  new  item  should  appear.  In  the 
example  shown  in  Fig.  32,  the  dataBlock  is  an  M 16  and  the  rotate  is  set  to 
“0”  or  false  (marked  with  a  red  arrow). 


[Apply  |  Name  i 

■  Transform 

position  -687.781  106.859  0.026C 

rotation  10  0  0 

scale  1  1  1 

LI  Misc 

jn  - 

nameTag 

dataBlock  MI6 

h. 

collidable  ID 

‘  .  W  IT  *$£*.:. '  ~  1 

static  HI 

rotate  □  r\ 

■  Dynamic  Fields  ** 

X 

Fig.  31  Example  of  adapting  animations:  spinning  M16 
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new  Item()  { 
position  = 
rotation  ■ 
scale  ■  "1 
dataBlocJc  = 


"-687. 781  106.859  0.0260671" 
"100  0"; 

1  1"; 

"M16"  ;| 


collidable  ■  "1" 
static  =  "1"; 
rotate  ■  "0";  ^ 


Fig.  32  Example  code  for  customizing  rotation  animations  on  the  M16  rifle 

6.1.6  Problem  No.  6:  Removing  Add-ons  from  Pre-loaded  Graphics  and 
Animations 

The  current  design  of  RIVET  includes  a  number  of  accessible  graphics  and 
animations.  However,  in  their  original  state  it  may  not  be  possible  to  directly 
integrate  those  graphics  into  a  new  mission.  For  example,  in  the  experimental 
design  using  the  TAC-C  vehicle  in  place  of  a  future  self-driving  vehicle  for  Soldier 
transport  in  a  noncombat  environment  (Section  5.3),  even  after  relocating  the 
camera  sensor,  it  was  still  possible  to  see  the  weapons  systems.  Since  the  design  of 
this  vehicle  was  built  in  multiple  pieces,  it  was  possible  to  simply  remove  the 
graphics  (i.e.,  the  weapons  systems)  while  maintaining  the  functionality  of  a  turret, 
from  which  the  camera  can  move  through  the  BOLT  interface.  Some  quick 
instructions  for  removing  the  weapons  systems  graphics  from  the  TAC-C  vehicle 
are  as  follows: 

1.  Open  the  directory:  C:\RIVET\sim.base.data\vehicles\tacc_weapons. 

2.  Select  the  following  4  .dts  media  files:  tumachinegunact; 
tu_maingun_act;  tu_rsta_act;  and  v_tac-c_weapons  (Fig.  33). 

3.  Do  not  delete  these  files.  Create  a  new  folder  in  this  directory  called  temp, 
and  place  the  files  in  this  folder  (Fig.  34)  to  allow  the  files  to  be  returned  to 
the  directory  at  a  later  time. 
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Organize  w  ^  f 

4  Favorites 
K  Desktop 
fc  Downloads 
i>  Recent  Places 

Libraries 
_>  Documents 
Music 
Pictures 
Videos 

As  Computer 
CL-  Local  Disk  (C:) 

Network 


4  items  s 


Burn  New  folder 

m  - 

Name 

Date  modified 

Type 

Size 

Old  Green  Version  of  tac-c_weapons 

11/3/2010  10:05  A... 

Adobe  Fireworks  P... 

344  KB 

placement 

11/3/2010  10:05  A... 

JPEG  image 

243  KB 

aU  smallTurret 

11/3/2010 10:05  A... 

VLC  media  file  (.dts) 

11  KB 

smallTurret 

11/3/2010  10:05  A... 

Adobe  Fireworks  P... 

4  KB 

*4  tac-c_weapons 

11/3/2010  10:05  A... 

Adobe  Fireworks  P... 

385  KB 

Za  tu machinegun 

11/3/2010 10:05  A... 

VLC  media  file  (.dts) 

22  KB 

Za  tu_machinegun_act 

11/3/2010 10:05  A... 

VLC  media  file  (.dts) 

14  KB 

Za  tu_maingun 

11/3/2010  10:05  A... 

VLC  media  file  (.dts) 

4  KB 

Za  tu_maingun_act 

11/3/2010  10:05  A... 

VLC  media  file  (.dts) 

5  KB 

Za  tu rsta 

11/3/2010 10:05  A... 

VLC  media  file  (.dts) 

17  KB 

Za  tu_rsta_act 

11/3/2010  10:05  A... 

VLC  media  file  (.dts) 

16  KB 

Za  tu_rsta_mast 

11/3/2010  10:05  A... 

VLC  media  file  (.dts) 

7  KB 

“S  v_gdrs_logo 

11/3/2010  10:05  A... 

Adobe  Fireworks  P... 

7  KB 

v_ladar_new 

11/3/2010 10:05  A... 

Adobe  Fireworks  P... 

430  KB 

v_tacc_b 

11/3/2010  10:05  A... 

Adobe  Fireworks  P... 

386  KB 

Za  v tac-c b tire 

11/3/2010  10:05  A... 

VLC  media  file  (.dts) 

14  KB 

Za  v tac-c weapons 

11/3/2010  10:05  A... 

VLC  media  file  (.dts) 

259  KB 

Za  v_tac-c_weapons_hull 

11/3/2010 10:05  A... 

VLC  media  file  (.dts) 

160  KB 

“S  v_vehicle_lights 

11/3/2010  10:05  A... 

Adobe  Fireworks  P... 

3  KB 

1  Date  modified:  11/3/2010  10:05  AM 

Date  created:  11/20/2014 11:18  AM 

□  d 


Fig.  33  Directory  location  of  files  to  remove  weapons  systems  from  the  TAC-C 


Fig.  34  Creation  of  a  temporary  folder 
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6.2  Support  and  Documentation 


6.2.1  RIVET  Developer's  Guide 

Perhaps  one  of  the  most  useful  and  relevant  sources  to  understanding  the  makeup 
and  architecture  of  RIVET  is  the  RIVET  Developer ’s  Guide  (General  Dynamics 
Robotic  Systems  2010).  This  document  is  included  in  the  software  and  provides  an 
introduction  to  the  TGE  (Chapter  1),  system  requirements  (Chapter  2),  a  high-level 
overview  of  RIVET  (Chapter  3),  instructions  for  creating  objects  (Chapters  4-7), 
instructions  for  world  building  and  introduction  to  the  editors  (Chapter  8), 
introduction  to  TorqueScript  (Chapter  9),  entity  AI  (Chapter  10),  dataset  creation 
(Chapter  11),  and  HITL  support  (Chapter  12-13).  However,  this  resource  is  a 
proprietary  document  that  may  also  be  difficult  to  understand  without  additional 
background  in  the  TGE,  game  development,  or  computer  programming. 

6.2.2  GarageGames.com  Online  Forums  and  Documentation 

Torque  Developer  Network  (tdn.garagegames.com)  is  the  official  reference  site  for 
coders,  artists,  or  anyone  working  with  Torque.  This  site  requires  a  login  and  is 
currently  in  its  beta  version.  It  has  a  number  of  tutorials  and  documentation  for  a 
wide  range  of  topics.  Since  all  materials  have  been  written  collaboratively  by 
Torque  developers,  it  can  sometimes  be  difficult  to  understand  by  a  novice  Torque 
user  as  it  has  varying  levels  of  descriptions  and  explanations.  Also,  some  of  the 
step-by-step  directions  reference  editors  that  are  not  available  in  the  RIVET  v.1.0. 
However,  the  documentation  often  provides  examples  of  TorqueScript  files  that  can 
be  leveraged  to  meet  a  user’s  needs  and  is  constantly  updated  with  new  information. 
In  addition,  a  number  of  resources  are  also  freely  available 
(http  ://docs  .garagegames .  com/tge/official/). 

6.2.3  The  Game  Programmer's  Guide  to  Torque 

The  Game  Programmer ’s  Guide  to  Torque  (Maurina  2006)  is  a  GarageGames  book 
that  provides  a  cogent,  base-level  introduction  to  Torque  from  start  (opening  the 
TGE  for  the  first  time)  to  finish  (putting  it  all  together).  The  core  substance  of  the 
book  focuses  on  the  game  elements  including  classes,  objects,  and  even  an 
introductory  glance  into  scripting  and  game  interfaces.  While  this  book  provides  a 
number  of  step-by-step  examples,  the  design  and  setup  of  RIVET  extends  beyond 
the  standard  layout  of  the  TGE.  In  addition,  it  may  not  go  into  the  level  of  detail 
needed  for  creating  HRI  missions  but  overall  is  a  valuable  resource. 
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6.2.4  Scripting 

While  this  report  does  not  provide  a  lot  of  detail  about  TorqueScript,  understanding 
the  “what”  and  the  “how”  can  add  customization  capabilities  to  the  VE.  To  begin, 
TorqueScript  is  a  programming  language  very  similar  to  C++.  Having  an 
introductory  understanding  of  programming  languages  can  help  immensely, 
especially  when  reading  different  resources.  This  report  provides  an  overview  of 
some  additional  resources  to  scripting.  A  general  introduction  to  TorqueScript  can 
be  found  by  reviewing  the  GarageGames  online  documentation.  This  information 
is  available  online  at  http://docs.garagegames.com/tgea/official/content 
/documentation/Scripting%20Reference/Introduction/TorqueScript.html.  For  a 
more  in-depth  review,  the  book  Advanced  3D  Game  Programming  All  in  One 
(Finney  2005)  comes  with  a  CD  and  provides  good  explanations  of  datablocks, 
vectors,  and  matrices,  along  with  an  entire  chapter  on  TorqueScript.  In  addition,  the 
book  3D  Game  Programming  All  in  One  (Finney  2007)  provides  an  introduction  to 
the  entire  process  of  creating  a  game  within  Torque,  including  building  the  models, 
creating  animations,  server-client  architectures,  creation  of  GUIs,  and  even  testing. 

6.2.5  Animations 

Chapter  7  of  the  RIVET  Developers  Guide  (General  Dynamics  Robotic  Systems 
2010)  provides  the  most  in-depth  instruction  as  to  creating  animated  models.  There 
are  multiple  choices  for  modeling  and  rendering  applications  such  as  Blender, 
Maya,  and  3ds  Max.  The  instructions  in  the  Guide  are  based  on  3ds  Max  by 
Autodesk. 

6.2.6  Other  Torque  Resources 

Finally,  other  online  resources  may  be  helpful  in  providing  guidance  and  instruction 
while  developing  specific  missions,  including  tutorials  by  Dr  David  J  Sushil:  A 
Torque  Game  Engine  Primer  (2008),  Simple  AI  in  the  Torque  Game  Engine  (2008), 
Creating  Doors  in  Torque  (2007),  Animating  Characters  in  Torque  (2007),  Sound 
Effects  in  Torque  (2007),  and  Controlling  Events  with  Triggers  in  Torque  (2007). 
These  resources,  as  well  as  a  number  of  examples  were  written  as  instructional 
guides  for  game  development  students.  These  step-by-step  guides  often  provide 
more  detail  than  programming  books  or  online  forums.  In  addition,  they  are  freely 
downloadable  .pdf  files  that  can  be  found  online  at  http://www.davidjsushil.com 
/index  .php?  action=tutorials . 
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7.  Conclusions 


Using  a  game  engine  (such  as  RIVET)  to  create  a  robotic  platfonn  application, 
while  including  a  separate  interface  for  human  interaction,  provides  a  division 
between  the  software  applications.  This  separation  allows  software  engineers  to 
make  changes  to  the  algorithms  in  perception,  intelligence,  and  mobility  while 
leaving  the  HRI  untouched.  Therefore,  development  of  each  part  may  continue 
while  still  allowing  for  testing  at  each  step.  The  game  engine  allows  for 
reproducibility  during  testing  to  capture  the  user  interactions  with  the  robot.  There 
is  no  longer  a  need  to  wait  for  the  final  product  to  roll  off  the  assembly  line  and  into 
the  hands  of  Soldiers  before  conducting  experimentation  to  explore  human 
interaction  questions.  Instead,  integrating  HRI  experimentation  early  on  can  help 
drive  design  choices  during  the  stages  of  development. 
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Appendix  A.  RIVET  Menus 
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Server  Computer:  Main  User.  The  first  button  on  the  main  menu  is  Load  Mission 
(Fig.  A-l). 


Fig.  A-l  RIVET  main  menu  GUI  for  main  user 


It  leads  the  user  to  the  next  GUI  where  the  user  can  choose  the  mission,  vehicle, 
and  whether  it  is  friendly  or  enemy  (Fig.  A-2).  The  image  on  the  right  is  the 
overhead  map  of  the  mission  area. 
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Fig.  A-2  RIVET  load  menu  GUI  for  main  user 
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Client  Computer:  Add  Additional  Players.  Up  to  64  players  can  join  a  mission 
(Fig.  A-3).  A  player  can  be  a  person  (e.g.,  Soldier),  vehicle  (e.g.,  Humvee),  robot 
(e.g.,  Talon),  or  a  sensor  attached  to  the  main  character  (from  the  Server  computer). 


Fig.  A-3  RIVET  main  menu  GUI  for  client  computer 

After  selecting  Join  Mission,  a  new  window  (Fig.  A-4)  will  open  that  will  allow 
selection  of  a  player  character  or  vehicle  [1]  and  side  (i.e.,  red,  blue,  neutral)  [2]. 
To  connect  to  the  correct  Mission,  click  Find  Servers  [3]  and  select  the  Server 
Name.  To  join  the  mission,  select  Join  Mission  [4]. 


Fig.  A-4  RIVET  join  mission  GUI  for  client  computer 
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Appendix  B.  CARVE  Menus 


Approved  for  public  release;  distribution  is  unlimited. 


59 


CARVE  Main  Menu.  The  Control  of  Autonomous  Robotic  Vehicle  Experiments 
(CARVE)  Main  Menu  (Fig.  B-l)  leads  the  user  to  a  series  of  submenus  located  in 
the  lower-right-hand  corner  of  the  screen.  These  include  the  Connection  button 
(bottom  right),  settings  submenu  (bottom  left),  user  interfaces  (top  left),  and  vehicle 
operations  submenu  (top  right).  These  4  icons  are  always  available  to  the  user  and 
are  accessed  by  a  mouse  interface. 
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l^][@ 

GENERAL.  DYNAMICS 

Robot*  Sytttms 

'\}m 

Fig.  B-l  CARVE  Main  Menu  GUI 
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Settings  submenu.  The  settings  submenu  has  a  few  options  for  customization  of 
the  connectivity,  video,  networking,  controller  choice,  and  exiting  the  simulation 
(Fig.  B-2). 


Fig.  B-2  Settings  submenu 

This  submenu  allows  the  experimenter  to  check  the  setup  of  the  experimental  space. 
This  includes  being  able  to  set  up  the  connectivity  of  the  sensor  and  vehicle  by 
entering  internet  protocol  (IP)  addresses  and  ports.  Connectivity  is  needed  to 
connect  CARVE  to  RIVET.  The  video  and  networking  buttons  were  designed  for 
later  versions  of  CARVE  so  that  the  video  and  bandwidth  can  be  degraded  to 
represent  poor  communication  situations.  The  controller  button  allows  the 
experimenter  to  set  the  device  that  will  be  used  in  the  mission  (e.g.,  joystick, 
steering  wheel).  Finally,  to  exit  the  program  is  self-explanatory. 
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User  Interfaces  submenu.  The  User  Interfaces  submenu  (Fig.  B-3)  provides  the 
experimenter  the  capability  to  set  up  a  number  of  additional  functionalities  for  the 
participant. 


Fig.  B-3  User  interfaces  submenu 

The  Video  button  allows  the  experimenter  to  set  the  main  screen  to  show  the  video 
stream.  The  Map  button  sets  the  main  screen  to  show  the  overhead  map  of  the  area. 
The  picture-in-picture  (PIP)  option  displays  either  the  map  or  video  opposite  to 
what  is  displayed  on  the  main  screen  in  a  smaller  screen  in  the  upper-right-hand 
comer  of  the  video  feed.  This  gives  the  user  additional  situation  awareness  during 
experiments.  The  Controls  button  provides  an  on-screen  option  for  controlling  the 
robot  or  vehicle.  While  the  Gesture  and  Speech  buttons  are  used  to  integrate  an 
Xbox  Kinect  sensor  into  an  experiment  if  naturalistic  communication  is  required. 
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Vehicle  Operations  submenu.  The  Vehicle  Operations  submenu  (Fig.  B-4)  allows 
the  user  to  change  the  posture  in  which  the  vehicle  is  used. 


Fig.  B-4  Vehicle  Operations  submenu 

Tele-op  (which  stands  for  teleoperation)  allows  the  user  to  drive  the  vehicle  with 
the  chosen  controller.  These  include  the  game  pad,  joystick,  or  racing  wheel,  to 
name  a  few.  If  the  vehicle  is  set  up  with  a  weapon  system,  the  Main  Gun  or  Machine 
Gun  settings  allow  the  movement  of  the  turret  with  the  same  controller.  The 
Waypoint  Follow  button  has  a  submenu  that  gives  the  user  the  control  to  initiate 
waypoint  following.  The  navigation  is  set  using  the  map  screen  to  create  a  waypoint 
path  for  the  vehicle  to  follow. 
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Intentionally  left  blank. 
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Appendix  C.  Creating  a  New  Mission:  Load  RIVET 
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1.  Open  RIVET.  The  Load  Mission  button  leads  the  user  to  the  next  graphical 
user  interface  (GUI)  shown  in  Fig.  C- 1  (see  also  the  RIVET  Developer ’s  Guide, 1 
p.  41-43).  It  allows  the  user  to  choose  the  mission,  the  player  character 
(vehicle/Soldier),  and  whether  it  is  friendly  or  enemy. 


Fig.  C-l  RIVET  load  mission  menu 


*  General  Dynamics  Robotic  Systems.  RIVET  developer’s  guide  (included  in  software  installer  package 
in  Docs  folder).  Westminster  (MD):  General  Dynamics  Robotics  Systems;  2010.  p.  41-43. 
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2.  Pick  a  player  character.  When  first  building  a  new  virtual  environment  (VE) 
for  HRI  research,  it  is  recommended  to  select  either  a  Soldier  or  Humvee  (Fig. 
C-2).  These  two  player  characters  are  designed  to  function  well  with  the 
keyboard  and  mouse  user  interface.  However,  when  using  RIVET  with  the 
BOLT  sensor  and  CARVE  application,  choose  only  a  vehicle  with  BOLT  listed 
in  the  title.  BOLT  deactivates  the  keyboard  and  mouse  associated  with  the 
Server  computer. 


Fig.  C-2  Selecting  a  player  character:  Humvee 
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3.  Choose  your  mission.  When  creating  a  new  VE,  it  may  be  beneficial  to  begin 
by  opening  a  Mission  file  that  has  already  been  created,  as  shown  in  Fig.  C-3 
(e.g.,  ARL  Test  World).  The  associated  TorqueScript  mission  file  (ARL  Test 
World.mis)  is  located  in  the  following  directory:  C:\RIVET\sim.base 
\data\missions. 


Fig.  C-3  Selecting  a  mission 

To  successfully  open  and  run  a  mission,  this  directory 
(C:\RIVET\sim.base\data\missions)  should  have  the  following  files:  Mission 
file  (.mis)  and  Terrain  file  (.ter).  To  run  BOLT  and  CARVE,  the  directory 
should  also  contain  a  Map  (.png)  and  a  location  file  (.cs). 
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4.  Launch  Mission.  Commands  for  controlling  the  player  character  and  switching 
camera  views  (first  person,  third  person,  or  birds-eye  view  [Fig.  C-4])  are 
available  in  the  RIVET  Developer’s  Guide 1  (p.  48-49). 


Fig.  C-4  Changing  camera  views:  example  of  a  birds-eye  view 


5.  Save  As  a  new  Mission  File.  Press  Fll  on  the  keyboard;  click  on  File,  and 
select  Save  Mission  As  (Fig.  C-5). 


Fig.  C-5  Flow  to  save  a  new  mission  file 
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6.  Rename  File.  This  will  save  the  mission  in  a  new  directory 
[C:\RIVET\sim.base\data\user_missions].  Make  sure  to  rename  the  file  (e.g., 
HRI  Experiment  l.mis);  otherwise  it  will  overwrite  a  previous  file  (Fig.  C-6). 


Fig.  C-6  Rename  file 

This  process  will  create  the  Mission  file  (.mis)  and  a  Terrain  file  (.ter).  To  check 
that  the  files  were  created,  open  the  directory  (C:\RIVET\sim.base 
\data\usermissions)  as  shown  in  (Fig.  C-7). 


Fig.  C-7  Save  mission  as  new  file  name 

7.  Save  Mission.  Once  you  have  created  a  new  Mission  file  with  a  new  name,  it 
is  important  to  save  often.  The  mission  can  be  saved  by  selecting  file  menu 
option  (Press  FI  1,  select  File,  and  select  Save  Mission).  It  is  also  possible  to 
save  the  mission  by  using  the  keyboard  command  [Ctrl+s]. 
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8.  Delete  Current  Objects  (e.g.,  building  and  path).  To  delete  an  object  use  the 
mouse,  hold  left  mouse  button  down  and  drag  the  mouse  to  select  the  area 
desired.  A  yellow  box  will  appear  around  the  objects  selected.  To  mark  that  the 
object  has  been  selected,  it  will  be  outlined  in  red  (Fig.  C-8).  To  delete  these 
objects,  press  the  Delete  key  on  the  keyboard.  Note  there  are  no  “undo”  options. 


Fig.  C-8  Selecting  objects  in  the  VE 

9.  Closing  RIVET.  There  are  different  ways  to  close  RIVET.  The  first  is  through 
the  File  menu  (FI  1,  File,  Quit).  The  second  is  by  using  the  keyboard  command, 
Alt+F4.  Remember  to  save  before  closing  RIVET. 
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10.  Open  Saved  Mission.  To  open  the  newly  created  User  Mission  file,  open 
RIVET  and  click  on  Launch  Mission  (see  step  1).  Click  on  the  User  Mission 
tab  and  select  the  mission.  As  shown  in  Fig.  C-9,  the  mission  name  will  be  the 
original  file  name  (e.g.,  ARL  Test  World)  even  though  the  file  was  saved  with 
a  new  name  [HRI  Experiment  l.mis].  This  is  still  the  newly  created  mission; 
however,  there  is  an  additional  step  to  change  the  name  of  the  file  in  the  Main 
Menu  GUI. 


Fig.  C-9  Open  saved  user  mission 
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1 1 .  Change  User  Mission  File  Name.  To  update  the  file  name  in  Main  Menu  GUI, 
close  the  RIVET  Main  Menu  (press  Cancel  and  Exit  to  close  RIVET.  The  next 
step  involves  editing  the  TorqueScript.  Open  the  Mission  file  (.mis)  from  the 
directory.  Directions  to  access  the  .mis  file  in  the  Data  Directory  can  be  found 
in  step  6.  .  Open  the  mission  file  by  using  Notepad,  Notepad++,  or  Torsion.  At 
the  top  of  the  .mis  file  is  the  TorqueScript  for  the  “name”  and  “descO”  (see  Fig. 
C-10  -  Original  Script).  Both  should  be  updated  so  that  name  matches  the  new 
User  Mission  name  and  descO  marks  the  person(s)  responsible  for  creating  the 
new  VE.  These  are  marked  in  red  text  (see  Fig.  C-10  -  Revised  Script). 


Original  script 

Revised  script 

new  ScriptObject(Missionlnfo)  { 

new  ScriptObject(Missionlnfo)  { 

name  =  “ARL  Test  World”; 

name  =  “RIVET  TR  Example:  HRI  Experiment  1”; 

descO  =  “General  Dynamics  Robotic  Systems”; 

descO  =  “Designed  by  Dr.  K.E.  Schaefer,  ARL”; 

aiPlayerFile  =  “testAIplayer.es”; 

aiPlayerFile  =  “testAIplayer.es”; 

descLines  =  “1”; 

descLines  =  “1”; 

map  =  “ARL  world”; 

}; 

map  =  “ARLworld”; 

}; 

Fig.  C-10  Example  TorqueScript  to  update  the  user  mission  name  for  the  main  menu  GUI 


12.  Now  when  RIVET  is  opened  and  Launch  Mission  selected,  the  User  Missions 
tab  will  list  the  updated  name  of  the  correct  user  mission  (Fig.  C- 11).  While  this 
will  not  inhibit  use  of  the  system,  updating  the  User  Mission  file  name  in  the 
Main  Menu  GUI  is  encouraged,  as  it  helps  to  accurately  and  effectively 
differentiate  between  multiple  User  Mission  files. 


Fig.  C-ll  Main  menu  GUI:  updated  user  mission  file  name 
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13.  Map.  Another  way  to  help  differentiate  between  the  user  missions  is  by  adding 
a  map  to  the  User  Mission  login  screen.  Map  creation  is  also  essential  when 
using  the  CARVE  application.  To  create  a  Map,  first  load  RIVET  and  launch 
the  User  Mission.  Next,  press  Alt+c  on  the  keyboard  to  switch  camera  view. 
Use  W,  A,  S,  D  keyboard  commands  at  the  same  time  as  moving  the  mouse  to 
move  upward  into  the  air. 

14.  Creating  the  Map.  To  initiate  the  process  for  creating  a  map,  first  follow  the 
steps  to  name  the  map  the  same  name  as  the  User  Mission.  Press  the  tilde  (~) 
key  on  the  keyboard  to  open  the  Console  and  type  $userPref::MissionMaps  = 
“HRI  Experiment  1,  4,  4,  0.4”;  as  shown  in  Fig.  C-12.  Next,  press  Enter  on  the 
keyboard  and  ~  to  close  the  Console.  To  create  the  Map,  press  Alt+m  on  the 
keyboard.  The  Map  (.png)  and  associated  location  file  (.cs)  are  automatically 
place  into  the  RIVET  directory  C:\RIVET  (Fig.  C-13). 


Fig.  C-12  Create  map  screen 


Fig.  C-13  RIVET  directory  location  for  map  (.png)  and  location  (.cs)  files 
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15.  Setting  up  a  Map  in  RIVET.  For  RIVET  to  be  able  to  successfully  read  the 
Map,  first,  move  the  map  and  location  files  (.png  and  .cs)  to  the  User  Mission 
directory  as  shown  in  Fig.  C-14  (C:\RIVET\sim.base\data\usennissions). 


Fig.  C-14  User  mission  directory 

Next,  open  up  the  Mission  file  (C:\RIVET\sim.base\data\usermissions\HRI 
Experiment  l.mis)  and  add  or  revise  the  TorqueScript  (see  Fig.  C-15,  line  6). 
This  will  now  call  the  Map.  When  RIVET  is  opened,  select  the  User  Mission, 
and  a  Map  will  now  be  present  (Fig.  C-16). 


1  (Globals)  | 

1 

// -  OBJECT  WRITE  BEGIN  - 

l] 

3  new  SimGroup (MissionGroup)  { 

3  new  ScriptObject (Missionlnfo)  { 

5 

aiPlayerFile  =  "HRIexperimentAIplayer.es"; 

6 

7 

name  =  "RIVET  TR  Example:  HRI  Experiment  1"; 

8 

descO  =  "Designed  by  Dr.  K.E.  Schaefer,  ARL"; 

9 

descLines  =  "1"; 

10 

}; 

11  ^ 

]  new  MissionArea (MissionArea)  { 

12 

Area  =  "-1032  -1016  2064  2048"; 

13 

flightCeiling  =  "300"; 

14 

f lightCeilingRange  =  "20"; 

15 

locked  =  "true"; 

16 

}; 

Fig.  C-15  TorqueScript:  adding  the  map 


Fig.  C-16  Main  Menu  GUI:  user  mission  file  with  map 
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16.  Map  Size.  To  check  the  map  size,  right  click  on  the  map  to  look  at  the  properties 
of  the  file.  Ideally,  the  map  should  be  1600  x  1600  pixels  and  under  2  MB  in 
size.  If  this  is  not  the  case,  use  an  outside  program  (e.g.,  Adobe  Fireworks)  to 
reduce  the  size  of  the  map  (.png).  If  this  step  is  missed,  RIVET  will  provide  an 
error  message  as  shown  in  Fig.  C-17. 


Fig.  C-17  Error  message 
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Appendix  D.  Creating  a  New  Mission:  Editing  the  Terrain 
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1 .  Terrain  Editor  Tool.  Use  the  Terrain  Editor  tool  to  adapt  the  height  and  shape 
of  the  terrain  (Fig.  D-l).  It  can  be  accessed  by  pressing  FI  1  on  the  keyboard 
and  selecting  Window  and  Terrain  Editor  from  the  drop-down  menu. 


Fig.  D-l  Terrain  Editor  tool 


Fig.  D-2  Terrain  Editor:  Action  submenu 
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3.  Brush  size.  Choose  the  size  of  the  brush  (Fig.  D-3). 


Fig.  D-3  Terrain  Editor:  Brush  submenu 

4.  To  edit  the  shape  of  the  terrain,  use  the  mouse.  Click  and  hold  the  left  mouse 
button  while  dragging  left,  right,  up,  or  down  (Fig.  D-4).  This  will  raise  or  lower 
the  ground  terrain. 


Fig.  D-4  Edit  the  shape  of  the  terrain 
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5.  Terrain  Painter  tool.  This  tool  bar  is  used  to  change  the  underlying  texture  of 
the  terrain.  Click  on  one  of  the  terrains  and  then  paint  the  environment  (Fig.  D-5). 


Fig.  D-5  Terrain  Painter  tool 


It  is  possible  to  add  additional  textures  by  clicking  on  Add  (Fig.  D-6,  right  side 
menu)  and  searching  through  the  menu  options  for  different  image  files  (.jpg) 
(Fig.  D-7).  These  image  files  can  also  be  found  in  the  terrains  directory, 
C:\RIVET\sim.base\data\terrains\terrains\snow2.jpg.  This  allows  for  greater 
flexibility  in  the  design  of  the  terrain  that  can  be  updated  at  any  time  during 
creation  of  the  virtual  environment. 


Fig.  D-6  Texture  submenu 
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Fig.  D-7  Available  terrain  textures 


6.  Save  often! 
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Intentionally  left  blank. 
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Appendix  E.  Add  Static  Objects 
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1 .  Change  Camera  View.  Currently,  the  virtual  environment  is  very  empty  (Fig. 
E-l).  Before  adding  objects,  it  is  important  to  change  the  camera’s  viewing 
angle. 


Fig.  E-l  Original  view 

To  toggle  your  camera  to  Fly  mode  (also  known  as  birds-eye  mode),  press  Alt+c 
on  the  keyboard.  This  same  action  can  also  be  achieved  through  the  menu 
options  (F 11)  by  selecting  the  Camera  menu  and  the  Toggle  Camera  option. 
After  the  camera  is  placed  in  Fly  mode,  pull  the  mouse  toward  you  and  hold 
down  the  “s”  key.  It  will  appear  that  you  are  “flying”  higher  into  the  air  while 
looking  down  at  the  ground.  It  should  be  possible  to  see  the  Humvee  on  the 
ground  below  (Fig.  E-2).  This  is  the  best  view  to  have  when  adding  static 
objects;  otherwise,  they  could  end  up  being  placed  underground. 


Fig.  E-2  Fly  mode 
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2.  Add  objects.  Use  the  World  Editor  Creator  to  add  objects  to  the  VE  (Fig.  E-3). 
To  open  the  World  Editor  Creator,  press  FI  1,  select  Window  and  World  Editor 
Creator.  It  may  also  be  possible  to  open  this  menu  by  pressing  F4  on  the 
keyboard. 


Fig.  E-3  World  Editor  Creator  tool  to  add  objects 

3.  Static  Objects.  RIVET  already  has  a  number  of  static  objects  available  for  use. 
Most  of  these  are  available  under  the  Interiors  or  Static  Shapes  directory 
(Fig.  E-4). 


Fig.  E-4  RIVET  static  object  directory 
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4.  Finding  Static  Objects.  It  may  be  difficult  to  know  what  static  objects  are 
currently  available.  RIVET  has  a  Mission  File  [3D  Object  Catalog  -  DO  NOT 
MODIFY! ! !]  that  provides  a  single  location  that  includes  a  number  of  (but  not 
all)  of  the  static  objects  available  (e.g.,  buildings,  roadways,  warehouse  objects, 
and  street  signs).  Fig.  E-5  depicts  the  Main  Menu  GUI  options  for  opening  the 
3D  Object  Catalog  file.  Figures  E-6  through  E-8  provide  examples  of  some  the 
static  objects  available. 


Fig.  E-5  3D  Object  Catalog  VE  located  in  the  Main  Menu  GUI 


Fig.  E-6  Options  for  static  buildings 

Approved  for  public  release;  distribution  is  unlimited. 


86 


Fig.  E-7  Options  for  static  warehouses 


Fig.  E-8  Options  for  static  signs  and  other  obstacles 
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5.  Locating  an  object’s  file  location.  To  find  the  location  of  an  object  in  the  tree 
structure  for  later  use  in  the  VE,  press  FI  1,  click  on  desired  object,  and  open 
the  World  Editor  Inspector  (Fig.  E-9).  The  World  Editor  Inspector  can  be 
opened  by  either  pressing  the  F3  key  on  the  keyboard,  or  pressing  F 1 1 ,  selecting 
Windows  and  World  Editor  Inspector  from  the  drop-down  menu. 


Fig.  E-9  World  Editor  Inspector  menu 
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6.  World  Editor  Inspector.  Click  on  open  file  dialog  box  (mouse  cursor,  outlined 
with  a  red  box)  and  then  search  for  the  name  of  the  object  (e.g., 
redBrick40sGarage)  in  the  list  (Fig.  E-10).  Make  a  note  that  this  building  is 
located  in  the  Interiors\DowntownDistrict  directory. 


Fig.  E-10  Locating  an  object  file  in  the  World  Editor  Inspector 

7.  Open  the  selected  mission  [RIVET  TR  Example:  HRI  Experiment  1],  and 
repeat  step  1  for  adding  static  objects.  The  screen  should  appear  as  shown  in 
Fig.  E-l  1. 


Fig.  E-ll  User  VE  from  fly  mode 
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Add  a  Static  Object.  Open  the  World  Editor  (FI  1)  and  then  open  the  World 
Editor  Creator  (see  steps  2  and  3).  Open  the  Interiors  menu  (Fig.  E-12). 


Fig.  E-12  Adding  a  static  object  through  the  Interiors  menu 

8.  Finding  the  Garage.  Scroll  down  to  find  DowntownDistrict  and  click  on 
redBrick40sGarage.  Once  the  garage  is  selected,  it  will  appear  in  the  VE. 
However,  as  shown  in  Fig.  E- 13,  the  building  is  located  on  top  of  the  Humvee 
and  partially  underground. 


Fig.  E-13  Initial  building  placement 
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9.  Moving  an  object.  To  move  an  object,  click  (on  the  X,  Y,  or  Z-axis)  and  drag 
the  building  to  a  desired  location  (Fig.  E-14). 


Fig.  E-14  Moving  an  object  within  the  VE 

10.  Rotating  an  object.  To  rotate  an  object,  press  Alt  on  the  keyboard  and  select 
the  Z-axis  with  the  mouse  (left  mouse  button).  While  holding  both  buttons, 
move  the  mouse.  The  object  will  rotate.  The  position,  rotation,  and  size  of  the 
object  can  also  be  changed  using  the  World  Editor  Inspector  (F3)  menu 
(Fig.  E-15). 


Fig.  E-15  Rotating  an  object 
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1 1 .  Keep  adding  objects  to  the  VE  until  it  resembles  the  experimental  environment. 
These  objects  are  automatically  added  to  the  Mission  file  (.mis).  An  example  of 
the  TorqueScript  that  is  added  into  the  .mis  file  for  a  static  object 
(redBrick40sGarage.dif)  is  provided  below. 

12.  Save  often! 

new  InteriorInstance()  { 

position  =  "-58.2412  -9.79301  -1.4"; 
rotation  =  "0  0  1  231.085"; 
scale  =  "1  1  1"; 

interiorFile  =  "~/data/interiors/DowntownDistrict/redBrick40sGarage.difi; 
useGLLighting  =  "0"; 
showTerrainlnside  =  "0"; 

}; 
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Appendix  F.  Creating  a  Path 
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1 .  Adding  a  path.  To  add  a  path  that  another  vehicle  or  person  may  follow,  open 
the  World  Editor  Creator  (F4).  Click  on  Path  to  name  the  path.  Assign  a  unique 
name  to  help  differentiate  between  the  paths.  In  Fig.  F-l,  the  path  name  is 
ExamplePath. 


Fig.  F-l  Naming  a  new  path 
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2.  Adding  a  PathMarker.  A  PathMarker  is  used  to  label  the  points  on  the  path 
(Fig.  F-2).  This  is  the  process  to  add  all  the  path  markers  needed  to  complete 
the  desired  path.  Make  sure  every  PathMarker  has  a  unique  name.  Label  names 
should  not  have  any  spaces  (use  underscore  if  needed).  Each  PathMarker  can 
be  moved  in  the  VE  in  the  same  way  static  objects  were  moved.  The  Y-axis 
should  face  the  direction  in  which  the  vehicle  is  to  move  for  each  PathMarker. 


Fig.  F-2  Adding  a  PathMarker  to  the  new  path 
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3.  Linking  the  PathMarkers.  Now  that  all  the  PathMarkers  are  placed  and 
rotated,  select  all  the  PathMarkers  from  the  tree  in  the  upper-right  corner 
(Folder  name  +MissionGroup  -  SimGroup)  (Fig.  F-3).  They  should  be  located 
at  the  bottom  of  the  tree  structure  since  they  were  the  most  recent  thing  added 
to  the  VE.  To  select  multiple  PathMarkers  at  the  same  time,  hold  down  the  Ctrl 
key  on  the  keyboard  and  click  on  each  marker  individually. 


Fig.  F-3  Selecting  all  PathMarkers  on  a  path 
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4.  Move  the  PathMarkers  under  the  path  name  (next  to  the  infinity  sign,  see 
Fig.  F-4).  In  the  VE,  all  markers  will  be  connected  by  a  dotted  green  line. 


Fig.  F-4  Linking  the  path 
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5.  To  ensure  the  vehicle  will  be  traveling  in  the  correct  direction,  check  and  fix 
the  sequence  order  (seqNum)  for  each  of  the  markers  (Fig.  F-5,  red  box).  Click 
on  the  sequence  number  in  the  Window  Editor  Inspector  (F3)  menu,  type  in  the 
correct  number,  and  click  Apply  (e.g.,  the  marker  named  ExamplePathl  should 
have  a  Sequence  number  of  1).  During  this  process,  it  is  normal  for  the  green 
dotted  line  to  look  askew  (Fig.  F-5).  It  will  become  smooth  again  when  all  of 
the  PathMarkers  are  set  to  the  appropriate  sequence  and  all  of  the  Y-axes  are 
facing  the  correct  direction. 


Fig.  F-5  Setting  the  path  sequence  order  (seqNum) 
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Appendix  G.  Add  a  Path-Following  Vehicle 
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1 .  Adding  a  path-following  vehicle:  Open  the  World  Editor  Creator  menu.  Select 
Shapes,  select  Vehicles,  and  then  choose  one  of  the  path-following  vehicles 
(e.g.,  PathFollowingPickupl)  (Fig.  G-l).  Highlight  the  vehicle  and  open  the 
World  Editor  Inspector  to  set  the  criteria  for  the  vehicle. 


Fig.  G-l  Path-following  vehicle:  setting  vehicle  criteria 

2.  Adapting  the  criteria.  In  the  World  Editor  Inspector  (F3),  give  the  truck  a 
name,  enter  the  PathName,  and  disableMove  (Fig.  G-2). 


Fig.  G-2  Path-following  vehicle:  adapting  criteria 
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3.  Adjust  the  vehicle’s  speed.  Right  now  the  truck  bounces  from  marker  to 
marker.  This  is  happening  because  the  truck’s  velocity  is  currently  set  to  zero. 
To  adjust  the  velocity  (Fig.  G-3),  click  on  each  marker  and  set  a  new  speed 
(meters  per  second).  Adjusting  the  speed  of  the  vehicle  at  each  marker  allows 
for  greater  flexibility  and  control  within  the  experiments. 


Fig.  G-3  Path-following  vehicle:  adjusting  the  vehicle  speed 
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Intentionally  left  blank. 
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Appendix  H.  Trigger  a  Non-player  Character 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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It  is  possible  to  add  a  non-player  character  (NPC)  to  this  Path  starting  at 
PathMarkerl  at  a  specific  time.  To  accomplish  this  task  requires  some 
programming  and  creation  of  three  new  TorqueScript  files,  which  we  named 
HRIexperimentAIplayer.es,  spawnNPC.es,  and  triggerNPC.es.  The  code  samples 
included  below  can  be  placed  directly  into  your  copy  of  RIVET.  Here,  we  provide 
the  TorqueScript  to  include  SoldierPlayerl  (the  animations  for  this  NPC  have 
already  been  included  in  a  previous  version  of  RIVET  and  can  be  located  in 
sim.base/data/shapes/soldier2  for  the  Soldiers,  sim.base/data/shapes/terrorists  for 
insurgents,  and  sim.base/data/shapes/player  for  the  civilians). 


1.  Initialize  the  NPCs  (Soldier  Player  1):  Create  a  new  TorqueScript  file.  Open 
a  text  file  and  add  the  following  code  block.  This  file  will  link  to  the  people 
animations  already  available  in  RIVET.  Save  as  a  unique  file  name  (e.g., 
HRIexperimentAIplayer.es)  in  the  following  directory 
C:\RIVET\sim.base\server.  This  file  initializes  the  NPC  and  allows  them  to  be 
spawned  on  a  path  and  to  move  on  a  path.  For  a  reference  point  also  look  at  a 
previously  created  Player  file  (simbase/server/aiPlayer.cs).  Green  text 
represents  comments  in  the  TorqueScript  code.  All  comments  are  also 
delineated  with  two  forward  slash  lines  at  the  beginning  of  the  line. 


// - 

//  Initializing  NPCs 

// - 

// . . 

//  AlPlayer  callbacks 

//  The  AlPlayer  class  implements  the  following  callbacks: 

// 

//  PlayerData:  :onStuck(%this,%obj ) 

//  PlayerData::onUnStuck(%this,%obj) 

//  PlayerData:  :onStop(%this,%obj ) 

//  PlayerData::onMove(%this,%obj) 

//  PlayerData::onReachDestination(%this,%obj) 

//  PlayerData::onTargetEnterLOS(%this,%obj) 

//  PlayerData::onTargetExitLOS(%this,%obj) 

//  PlayerData::onAdd(%this,%obj) 

// 

//  Since  the  AlPlayer  doesn't  implement  its  own  datablock,  these  callbacks 
//  all  take  place  in  the  PlayerData  namespace. 

//- - 

function  BasePlayer:  :onReachDestination(%this,%obj )  { 
if  (%obj.crowd  !$=  "")  { 

%endPath  =  %obj.crowd.isEndOfPath(%obj  .index); 
if(%endPath)  { 

AIPlayer::decideAction(%obj); 

} 

else  { 

AlPlayer:  :nextPathDestination(%obj ); 


else  { 

//echo(  "AlPlayer: :onReachDestination  warning  -  Crowd  is  blank!" ); 


if  (%obj.path  !$=  "")  { 

if  (%obj.currentNode  =  %obj  .targetNode) 
%this.onEndOfPath(%obj,%obj.path); 
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else 

%obj .  moveT  oN  extNodeQ ; 


} 

else  { 

//echo(  "AIPlayer::onReachDestination  warning  -  Path  is  blank!" ); 

} 


function  BasePlayer::onIdleOver(%this,%obj)  { 
if  (%obj. crowd  !$=  "")  { 

AiPlayer:  :decideAction(%obj ); 


else  { 

//echo(  "AiPlayer: :onidleOver  warning  -  Crowd  is  blank!" ); 


} 

function  BasePlayer:  :onMoveStuck(%this,%obj){ 
if  (%obj. crowd  !$=  "")  { 

AiPlayer:  :stuck(%obj ); 

} 

else  { 

//echo(  "AiPlayer: :onMoveStuck  warning  -  Crowd  is  blank!" ); 


} 

function  BasePlayer::onEndOfPath(%this,%obj,%path)  { 
%obj.nextTask(); 

//  %obj.moveToNode(0); 

} 

function  BasePlayer::onEndSequence(%this,%obj,%slot)  { 
echo("Sequence  Done!"); 

%obj.stopThread(%slot); 

%obj.nextTask(); 

} 

// - 

//  SoldierPlayerl 

//  Add  this  section  for  all  NPCs  that  you  have  animations  available 

jj _ 

datablock  PlayerData(SoldierPlayerl  :  SoldierBodyTypel)  { 
shootingDelay  =  2000; 

}; 


function  SoldierPlayerl  ::onReachDestination(%this,%obj)  { 
BasePlayer::onReachDestination(%this,%obj); 

} 

function  SoldierPlayerl ::onIdleOver(%this,%obj)  { 
BasePlayer:  :onIdleOver(%this,%obj ); 

} 

function  SoldierPlayerl ::onMoveStuck(%this,%obj)  { 
BasePlayer:  :onMoveStuck(%this,%obj); 

} 

function  SoldierPlayerl ::onEndOfPath(%this,%obj,%path)  { 
BasePlayer:  :onEndOfPath(%this,%obj,%path); 

} 

function  SoldierPlayerl ::onEndSequence(%this,%obj,%slot)  { 
BasePlayer:  :onEndSequence(%this,%obj,%slot); 

} 
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// - 

//  AlPlayer  static  functions 

function  AlPlayer:  :spawn(%datablock,  %name,  %spawnPoint) 

{ 

%player  =  new  AIPlayer()  { 
dataBlock  =  %datablock; 
path  = 

}; 

echo("AIPlayer::spawn  -  Player:", %player); 

MissionCleanup.add(%player); 

%player.setShapeName(%name); 

%player.setName(  %name ); 

%player.setTransform(%spawnPoint); 
echo("AIPlayer::spawn  -  spawnPoint:",%spawnPoint); 

return  %player; 

} 

function  AlPlayer::spawnOnPath(%datablock,  %name,  %path)  { 

//  Spawn  a  player  and  place  him  on  the  first  node  of  the  path 
echo("AIPlayer::spawnOnPath  -  Path:",%path); 
if  (!isObject(%path))  { 

echo("AIPlayer::spawnOnPath  -  Cannot  Find  Path!"); 
return; 

} 

%node  =  %path.getObject(0); 
echo("AIPlayer::spawnOnPath  -  Node:",%node); 

echo("AIPlayer::spawnOnPath  -  Node  Transform:", %node.getTransform()); 
%player  =  AIPlayer::spawn(%datablock,  %name,%node.getTransform()); 
echo("AIPlayer::spawnOnPath  -  Player:", %player); 

return  %player; 

} 

// - 

//  AlPlayer  methods 


function  AlPlayer:  :followPath(%this,%path,%node)  { 

//  Start  the  player  following  a  path 
echo("AIPlayer::followPath  -  Entered  method.\n"); 

%this.stopThread(0); 

if  (!isObject(%path))  { 

%this.path  = 

echo("AlPlayer::followPath  -  Attempting  to  follow  bad  path.Vn"); 
return; 

} 


if  ((%node  >  %path.getCount()  -  1)) 

%this.targetNode  =  %path.getCount()  -  1; 
else 

%this.targetNode  =  %node; 

if  (%this.path  $=  %path) 

%this.moveToNode(%this.currentNode); 
else  { 

%this.path  =  %path; 
%this.moveToNode(0); 

//%this.currentNode  =  0; 

} 
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function  AlPlayer:  :moveToNextNode(%this) 

{ 

if  (%this.targetNode  <  0  ||  %this.currentNode  <  %this.targetNode)  { 
if  (%this.currentNode  <  %this.path.getCount()  -  1) 
%this.moveToNode(%this.currentNode  +  1); 
else 

%this.moveToNode(0); 

} 

else 

if  (%this.currentNode  =  0) 

%this.moveToNode(%this.path.getCount()  -  1); 
else 

%this.moveToNode(%this.cuiTentNode  -  1); 

} 

function  AlPlayer:  :moveToNode(%this,%index) 

{ 

//  Move  to  the  given  path  node  index 
%this.currentNode  =  %index; 

%node  =  %this.path.getObject(%index); 

%this.setMoveDestination(%node.getTransfonn(),  %index  ==  %this.targetNode); 

} 


function  AlPlayer:  :pushTask(%this,%method)  { 
if  (%this.tasklndex  $= ""){ 

%this.taskhidex  =  0; 

%this.taskCurrent  =  -1; 

} 

%this.task[%this.tasklndex]  =  %method; 

%this.tasklndex++; 
if  (%this.taskCurrent  =  -1) 

%this.executeTask(%this.taskhidex  -  1); 

} 

function  AlPlayer:  :clearTasks(%this)  { 

%this.tasklndex  =  0; 

%this.taskCurrent  =  -1; 

} 

function  AlPlayer:  :nextTask(%this)  { 
if  (%this.taskCurrent  !=  -1) 

if  (%this.taskCurrent  <  %this.tasklndex  -  1) 

%this.executeTask(%this.taskCurrent++); 

else 

%this.taskCurrent  =  - 1 ; 

} 

function  AlPlayer:  :executeTask(%this,%index)  { 

%this.taskCurrent  =  %index; 
eval(%this.getld()  @  @  %this.task[%index]  @ 

} 

// - 

function  AlPlayer:  :singleShot(%this) 

{ 

//  The  shooting  delay  is  used  to  pulse  the  trigger 
%this.sethnageTrigger(0,true); 

%this.sethnageTrigger(0,false); 

%this.trigger  =  %this.schedule(%this.shootingDelay,singleShot); 

} 

function  AlPlayer:  :wait(%this,%time)  { 

%this.schedule(%time  *  1000,"nextTask"); 

} 

function  AlPlayer:  :done(%this,%time)  { 

%this.schedule(0,  "delete"); 

} 
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function  AIPlayer::fire(%this,%bool)  { 
if  (%bool)  { 

cancel(%this.trigger); 

%this.singleShot(); 

} 

else 

cancel(%this.trigger); 

%this.nextTask(); 

} 

function  AlPlayer:  :aimAt(%this,%object)  { 
echo("Aim:  "  @  %object); 

%this.setAimObject(%object); 

%this.nextTask(); 

} 

function  AlPlayer:  :animate(%this,%seq)  { 
echo("Set  animation:  "  @  %seq); 

//%this.stopThread(0); 

//%this.playThread(0,%seq); 

%this.setActionThread(%seq); 

} 

return  %player; 

2.  In  the  same  directory,  open  game.cs  and  add  an  executable  (exec)  function  to 
this  file  around  lines  190-200  exec("./HRIexperimentAIplayer.cs");  This  will 
allow  the  new  file  to  be  executed  during  the  Mission  run.  Please  note  that  the 
exec  function  is  colored  purple.  If  you  use  Torsion  IDE  for  developing  code,  all 
exec  functions  are  purple  (refer  to  pg.  38  for  more  infonnation  on  Torsion). 


3.  Open  the  Mission  file  HRIExperimentl.mis  (located  in 
C:\RIVET\sim.base\data\user_missions)  and  update  the  initial  mission  info: 
new  ScriptObject(Missionlnfo).  This  allows  the  Mission  File  to  call  to  the 
Alplayer  information. 


new  ScriptObject(Missionlnfo)  { 

name  =  "RIVET  TR  Example:  HRI  Experiment  1"; 
descO  =  "Designed  by  Dr.  K.E.  Schaefer,  ARL"; 
aiPlayerFile  =  "HRIexperimentAlplayer.es"; 
descLines  =  "1"; 
map  =  "ARLworld"; 

h _ 
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4.  Add  TorqueScript  File  to  Spawn  a  NPC:  Add  the  following  code  block  into 
a  text  file  and  save  as  spawnNPC.es  in  the  following  directory 
C:\RIVET\sim.base\server.  This  will  allow  NPCs  to  be  created  by  a  trigger  file. 

StickNumber  =  0; 

STotalExposureTime  =  0; 

//Hook  into  the  mission  editor . 

function  spawnNPCTrigger::create(%data)  { 
echo("spawnNPCTrigger::create"); 

//  The  mission  editor  invokes  this  method  when  it  wants  to  create  an  object  of  the  given  datablock  type. 

%obj  =  new  spawnNPCTrigger()  { 
dataBlock  =  %data; 

}; 

return  %obj; 

} 

//  dummy  datablock  needed  to  create  a  trigger 
//  the  values  in  this  datablock  are  not  used 

datablock  TriggerData(spawnNPCTrigger)  { 
tickPeriodMS  =  100; 

}; 


//Events - 

//this  event  is  fired  from  the  mission  editor  when  a  spawnNPCTrigger  is  added 

function  spawnNPCTrigger:  :onAdd(%this,%obj)  //this  event  is  fired  from  the  mission  editor 

{ 

Parent::onAdd(%this,%obj); 
echo("spawnSoldierTrigger:  :onAdd" ); 

%size  =  %this.triggerRadius; 

%scale  =  VectorScale(%obj.  scale,  %size*2); 

%sizeVec  =  %size  @"  "@  %size  %size; 

%pos  =  VectorAdd(%obj.position,%sizeVec); 

//  now  we  have  to  give  the  static  shape  a  controlling  trigger  of  the  right  size  and  location 
%trigger  =  new  Trigger/)  { 

dataBlock  =  spawnSoldierTrigger; 
position  =  %pos; 
rotation  =  %obj  .rotation; 
scale  =  %scale; 

polyhedron  =  "0.000000  1.0000000  0.0000000  1.0000000  0.0000000  0.0000000  0.0000000  -1.0000000 
0.0000000  0.0000000  0.0000000  1.0000000"; 

}; 

%trigger.setActive(true); 

%trigger.  setO  wner(%obj ); 

%obj .  setTrigger(%trigger) ; 

} 

//this  event  is  fired  when  an  object  enters  the  trigger 

function  spawnNPCTrigger:  :onEnterTrigger(%this,%trigger,%obj ) 

{ 

if(strcmp(%obj  .getName(),  "userl ")  =  0)  { 

echo("spawnNPCTrigger::onEnterTrigger  spawning  "  @  %trigger.npcType  @  "  on  path  "  @ 
%trigger.npcPath); 

echo("Vehicle  name  "  @  %obj.getName() ); 

%player  =  AlPlayer:  :spawnOnPath(%trigger.npcType,"SoldierNPC",%trigger.npcPath); 
%player.followPath(%trigger.npcPath,-l); 

%player.setMoveSpeed(0.4);//m/s 

//  %player.setInventory(M4, 1 ); 

//  %player.setInventory(M4Ammo,  1 00); 

//  %player.mountImage(M4Image,0); 

} 

} 
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5.  In  the  same  directory,  open  game.cs  and  add  an  exec  function  to  this  file  around 
lines  190-200  exec("./spawnNPC.cs");  This  will  allow  the  new  file  to  be 
accessed  during  the  Mission  run. 

6.  Add  a  Trigger  to  the  Mission  File:  A  trigger  will  be  represented  by  a  pink 
box  in  the  VE.  When  your  Player  Character  (e.g.,  Soldier  or  vehicle)  crosses 
through  the  Trigger  point,  it  will  initiate  the  Spawn  function  of  the  NPC  at  Path 
Marker  1 .  Open  Mission  File  and  add  this  code. 

new  Trigger(NPCl)  { 

position  =  "-12.899  44.4245  0.8"; 
rotation  =  "00  1  213.805"; 
scale  =  "103  2"; 

dataBlock  =  "spawnNPCTrigger"; 

polyhedron  =  ”0.0000000  0.0000000  0.0000000  1.0000000  0.0000000  0.0000000  0.0000000  -1.0000000 

0.0000000  0.0000000  0.0000000  1.0000000"; 

onGroup  =  "Default  Value"; 

npcType  =  "SoldierPlayerl"; 

npcPath  =  "MissionGroup/ExamplePath"; 

}; 


NPC1:  This  is  the  name  we  provided  to  this  Non-Player  Character.  You  can 
edit  this  to  be  a  unique  name. 

Position:  This  is  the  location  of  where  the  Trigger  will  be  located  in  the  VE.  A 
good  starting  point  is  either  near  the  initial  SpawnSphere  or  near  your 
PathMarker  1 .  Y ou  will  be  able  to  move  it  to  wherever  you  need  when  you  open 
the  VE. 

Rotation:  This  is  the  orientation  of  the  Trigger.  It  is  adaptable  within  the  VE. 

Scale:  This  is  the  size  of  the  Trigger.  It  is  adaptable. 

dataBlock:  This  is  what  calls  to  the  spawnNPC.es  file  to  create  the  NPC. 

npcType:  This  tells  the  spawnNPC.es  which  character  you  want  to  create.  **If 
you  directly  copy  these  TorqueScript  files  into  your  version  of  RIVET,  you  will 
only  have  access  to  SoldierPlayerl. 

npcPath:  This  lets  you  set  which  path  you  want  your  NPC  Soldier  to  be 
located.  It  is  adaptable. 

**  If  you  want  to  trigger  more  Soldiers  to  this  same  path,  all  you  need  to  do  is  add 
the  code  from  Step  #6  to  the  Mission  file  and  move  the  location  of  the  Trigger  to 
meet  your  needs  for  the  experimental  Design. 
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Appendix  I.  Delete  a  Non-player  Character  with  a  Trigger 
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1 .  Delete  a  NPC:  There  may  be  times  that  you  want  to  delete  the  NPC.  Add  the 
following  code  to  the  SpawnNPC.es  fde 


// - 

//Delete  NPC  at  a  Trigger 
// . . 

datablock  TriggerData(deleteNPCTrigger) 

{ 

tickPeriodMS  =  100; 

}; 

function  deleteNPCTrigger:  :onEnterTrigger(%this,%trigger,%obj ) 

{ 

if(isObject(SoldierNPC)  ||  isObject(Civilian)) 

{ 

echo("deleteNPCTrigger::onEnterTrigger  deleting  "  @  %obj.getname()); 
%obj.applyDamage(  10000); 

//  %obj.delete(); 

} 

} 


2.  Add  a  Delete  Trigger  to  the  VE:  Add  the  following  code  to  the 
HRIExperimentl.mis  file  to  create  the  Delete  NPC  trigger 

new  Trigger(delete_NPC  1 )  { 

position  =  "-36.64 11  -71 .5004  0.4"; 
rotation  =  "0  0-1  88.8997"; 
scale  =  "5  3  2"; 

dataBlock  =  "deleteNPCTrigger"; 

polyhedron  =  "0.0000000  0.0000000  0.0000000  1.0000000  0.0000000  0.0000000  0.0000000  -1.0000000 
0.0000000  0.0000000  0.0000000  1.0000000"; 

}; 
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Appendix  J.  Creating  Crowds 
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1.  Open  the  Crowd  Editor.  Double-click  the  CrowdEditor.exe  file  located  in 
C:\RIVET\support\Crowd  Tool. 

2.  Load  the  Map.  The  map  is  loaded  by  selecting  File  from  the  menu  in  the  upper- 
left-hand  comer  of  the  application  and  choosing  Load  Map  (Fig.  J-l).  This  will 
bring  up  a  dialog  box  to  select  the  map  file  from  the  computer. 


Fig.  J-l  Crowd  Editor  menu 

3.  Add  a  Crowd.  Click  Add  in  the  upper-right  comer  (Fig.  J-2).  Another  GUI  will 
open  where  the  name  of  the  crowd  is  entered.  Currently,  for  the  software  to 
work  properly,  the  crowds  must  be  named  Crowdl ,  Crowd2,  . . .  Crowd/;,  where 
n  is  the  number  for  the  last  Crowd  in  the  scene.  This  will  generate  information 
in  the  crowd  list. 


Fig.  J-2  Add  a  crowd 
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4.  Adjust  the  Crowd’s  Properties.  The  default  settings  will  be  used  in  the 
properties  menu  unless  the  destination  count,  people  count,  and  percent  idle  are 
changed.  The  crowds  are  grouped  in  tens  and  each  person  type  can  be  selected 
out  of  the  possible  choices  (Fig.  J-3). 


Fig.  J-3  Crowd  properties  menu 
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5.  Define  the  Location  for  the  Crowd.  Choose  the  size  of  the  brush  using  the 
Painting  slider  (Figs.  J-4  and  J-5).  This  will  adjust  the  area  of  the  brush.  The 
outline  choice  is  either  a  rectangle  or  a  circle,  which  is  the  default.  The  Erase 
Radio  button  is  used  to  undo  any  painting  previously  done. 


Fig.  J-4  Crowd  Location  menu 


Fig.  J-5  Crowd  location  area 

Approved  for  public  release;  distribution  is  unlimited. 


116 


6.  Generate  the  Contour.  The  tolerance  for  the  contour  generation  is  set  to  5.0. 
By  clicking  on  Generate  Contours,  the  application  will  outline  the  area  set  for 
the  crowd  (Figs.  J-6  and  J-7). 


Fig.  J-6  Generate  Contour  menu 


Fig.  J-7  Contour  area 
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7.  Save  Mission.  Choose  Save  Mission  from  the  menu  and  then  find  the  mission 
file  (.mis)  with  which  this  crowd  will  be  associated.  It  will  ask,  “Do  you  want 
to  overwrite?”  Click  yes  and  it  will  add  the  following  to  the  end  of  the 
TorqueScript  file.  These  are  the  parameters  set  in  the  GUI.  The  positions  are 
the  corners  of  the  boundary  line  around  the  contour,  which  will  enable  RIVET 
to  create  the  crowd  area  inside  the  application. 

new  Crowd(Crowdl)  { 

PeopleCount  =  "10"; 

DestinationCount  =  "10"; 

IdlePercent  =  "0.25"; 

IdleTime  =  "1  2"; 

AvailablePersonfO]  =  "CivMale"; 

AvailablePersonfl]  =  "CivFemale"; 

AvailablePerson[2]  =  "MalePlayerl"; 

AvailablePerson[3]  =  "MalePlayer2"; 

AvailablePerson[4]  =  "FemalePlayerl"; 

AvailablePerson[5]  =  "FemalePlayer2"; 

AvailablePerson[6]  =  "FemalePlayer3"; 

AvailablePerson[7]  =  "FemalePlayer4"; 

AvailablePerson[8]  =  "SoldierPlayerl"; 

AvailablePerson[9]  =  "SoldierPlayer2"; 

Positions  =  "10.8  3.2,11.2  -25.2,16.4  -25.2,16.4  -3,24  -2.4,23.6  3.2"; 

}; 


Once  a  mission  is  complete,  there  is  always  an  opportunity  to  make  changes  by 
choosing  File,  Load  Crowds  from  the  main  menu  and  selecting  the  mission  to 
change.  This  selection  will  then  read  in  the  crowd  details  in  the  GUI  and  allow 
changes  to  any  part  of  the  crowd.  Ensure  that  Generate  Contours  is  selected  before 
saving  if  the  contour  is  repainted. 


Approved  for  public  release;  distribution  is  unlimited. 


118 


8.  Example.  An  example  of  the  way  to  script  a  file  for  crowds  is  in  the 
crowdAIplayer.es  file.  Once  the  mission  is  launched,  the  crowds  will  appear. 
An  overhead  view  from  the  World  Editor  in  Fig.  J-8  shows  3  crowds  labeled  1, 
2,  and  3.  The  blue  squares  are  the  contour  points  noted  in  the  mission  file.  The 
green  lines  are  contour  boundaries  while  the  red  squares  are  the  destination 
points  to  which  the  crowd  participants  will  travel.  These  points  are  chosen 
randomly  for  each  AI  player  entity. 


Fig.  J-8  Overhead  view  of  multiple  crowds  in  World  Editor 
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Intentionally  left  blank. 
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Appendix  K.  Loading  the  CARVE  Application 
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1.  Setup.  Prior  to  opening  CARVE,  check  to  make  sure  that  both  the  RIVET 
Server  computer,  and  the  RIVET  BOLT  computer  user  mission  directories 
include  the  Mission  File  (.mis),  Terrain  File  (.ter),  Map  (.png),  and  associated 
.cs  file.  Specific  directions  on  how  to  build  these  files  are  in  Appendix  B. 


Fig.  K-l  Required  files 


2.  Start  RIVET  Server.  Click  on  RIVET  shortcut.  Set  vehicle  to  be  a 
BOLT-enabled  vehicle,  select  the  User  Mission,  and  click  Launch  Mission  (Fig. 
K-2). 


Fig.  K-2  Start  RIVET  Server  computer  with  BOLT-enabled  vehicle 
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3.  Start  RIVET  Client.  Mount  the  BOLT  sensor.  This  option  allows  users  to 
BOLT  an  additional  sensor  on  the  vehicle  and  is  required  if  the  user  will  be 
using  the  CARVE  interface.  After  selecting  the  Mount  Sensor  button,  a  new 
GUI  menu  will  open.  At  this  time,  check  to  make  sure  the  vehicle  name  is  set 
to  user  1  [1],  and  that  the  sensor  selected  is  the  BOLT  sensor  [2].  Next,  click  on 
Find  Servers  [3],  select  the  correct  IP,  and  select  Mount  Sensor  [4]  (Fig.  K-3). 


Vehicle  Name:  userl 


E 


One  server  found. 


Server  Name 


Sensor: 


BOLT  Sensor 


Connections 


iy64 


RIVET  IP:  192.168.1.10 


Cancel 


Find  Severs 


Mount  Sensor 


Refresh  Server  List 


Fig.  K-3  Mount  the  RIVET  BOLT  sensor  to  the  vehicle  using  the  Client  computer 
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It  is  also  possible  to  customize  the  sensors  through  the  sensor  drop-down  menu 
(Fig.  K-3,  [2]).Vehicles  are  equipped  with  various  sensors  that  are  configurable 
through  the  Sensor  Configuration  GUI  (Fig.  K-4). 


Sensors  Configurations: 


back 

BOLT  local  Sensor 
BOLT  PM 
BOLT  Sensor 
Camera  Plus 
front 

Front  UDP  camera 
KBOT PM 

ipft 


1 


Delete  Sensor 


Configuration  name:  |bact 
Sensor  Type: 

Position:  |-0.5  0.0  -2.5 
Parameters:  | 


=□ 


Remote  mountable 


Orientation:  |0.0  -2.0180.0 


Fig.  K-4  Sensor  Configuration  GUI 


The  vehicles  are  selected  in  the  drop-down  box  (Fig.  K-5).  These  are 
configurable  prior  to  the  load  mission  menu  using  vehicle  configuration  GUI. 
Sensors  that  were  configured  are  added  to  each  vehicle  configuration  with  a 
given  position  and  orientation. 


Vehicle  Configurations: 

Soldier 

Stryker 

T2  with  SMI  interface 

TAC-C  Cargo 

TAC-C  Command 

J 

TAC-C  Weapon 

TAC-C  Weapon  with  BOLT  PM 

▼  1 

Delete  Vehicle 


Configuration  name:  |TAC-C  Weapon  with  BOLT  PM  Vehicle: 
Sensors  and  capacities: 


Fig.  K-5  Vehicle  configuration 
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Sensors  can  be  local  to  the  server  or  loaded  on  the  Client  machine  and  added 
through  the  Mount  Sensor  GUI  (Fig.  K-6). 


Vehicle  Name:  |usei  (l 


Fig.  K-6  Mount  Sensor  GUI 


4.  Open  CARVE.  Now  that  both  the  RIVET  server  and  RIVET  client  BOLT 
sensor  computers  are  activated,  click  on  the  CARVE  shortcut  icon  (Fig.  K-7). 


Fig.  K-7  CARVE  shortcut  icon 

5.  Check  Connectivity.  Check  connectivity  to  make  sure  that  CARVE  can  “talk” 
to  the  RIVET  Server  and  Client  machines  (Figs.  K-8  and  K-9). 


Fig.  K-8  CARVE  Connectivity  tool 
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CARVE 


Sensor  IP  192.168.1.11  Port  H29 

Vehicle  IP  192.168.1.10  Port  1128 


GENERAL  DYNAMICS 

Robot  c  Systems 


Fig.  K-9  Check  connectivity  with  RIVET  machines 


6.  Connect  CARVE  to  RIVET.  During  this  initialization  phase,  CARVE  will  be 
receiving  the  map  from  RIVET  (Fig.  K-10).  This  can  take  upwards  of  a  minute. 


CARVE 

* 

m  t  ■■■» 

4  'r 

receiving  map... 

|  tele-op 

Speed:  0.0 

GENERAL  DYNAMICS 

Robot.:  Systems 

Fig.  K-10  Map  upload  from  RIVET  to  CARVE 
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7.  Failure  to  Open.  If  CARVE  is  unable  to  receive  the  map,  it  will  provide  an 
error  message.  At  this  point,  follow  the  directions  in  Section  6. 1  to  identify  the 
issue.  If  CARVE  successfully  loaded,  the  video  sensor  will  appear  in  the  main 
screen  window. 


CARVE 

u  'em 

A  (gl 

[  ^  tele-op 
Speed:  0.0 

GENERAL  DYNAMICS 

Robot  c  Systems 

\ 

Fig.  K-ll  Example  of  the  CARVE  failure  to  load  map  screen 

8.  Saving  the  Map.  For  faster  future  loading,  it  is  important  to  save  the  Map.  First, 
open  the  map  (Fig.  K-12). 


Fig.  K-12  Saving  the  map  in  CARVE:  open  the  map 


Approved  for  public  release;  distribution  is  unlimited. 


127 


VI0EIB 


9.  Select  Map  Save  Icon.  Once  the  map  is  opened,  choose  the  select  map  save 
icon  (Fig.  K-13). 


Fig.  K-13  Saving  the  map  in  CARVE:  select  map  save  icon 


10.  Save  Map:  When  saving  the  map,  use  a  unique  Map  name  and  write  it  down 
because  it  is  used  when  creating  a  CARVE  Mission-specific  shortcut  (Fig.  K- 
14). 


Fig.  K-14  Saving  the  map  in  CARVE:  assign  a  unique  map  name 
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11.  Close  CARVE.  Exit  the  program  by  selecting  the  exit  program  icon  (Fig.  K- 
15). 


Fig.  K-15  Saving  the  map  in  CARVE:  exit  the  program 


12.  Create  CARVE  Mission  Shortcut.  This  shortcut  pre-loads  the  map  and 
reduces  wait  time  (Fig.  K-16).  On  the  desktop,  right  click  on  the  CARVE 
shortcut,  and  select  Create  Shortcut.  Rename  the  new  CARVE  shortcut. 


Open 

Troubleshoot  compatibility 
Open  file  location 
*V  Run  as  administrator 
7-Zip 

•5  Scan  with  Microsoft  Security  Essentials... 
Pin  to  Taskbar 
Pin  to  Start  Menu 


Restore  previous  versions 
Send  to 


Cut 

Copy 


Create  shortcut 

Delete 

Rename 

Properties 


Fig.  K-16  Create  a  CARVE  shortcut  for  a  specific  mission 
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13.  Edit  Properties.  Right  click  on  the  new  shortcut,  and  select  properties.  Add  a 
space  and  /map:Maps\HRIexperimentl.rmap  (or  whatever  you  named  your 
map)  to  the  end  of  the  Target  (Fig.  K-17).  Now,  the  map  will  instantaneously 
load  when  selecting  the  new  CARVE  shortcut. 


Fig.  K-17  Naming  the  new  CARVE  shortcut  with  the  specific  mission  map 
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Appendix  L.  CARVE  Directions  for  Waypoint  Following 
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1 .  Adding  Waypoints  for  Semi- Autonomous  Navigation.  Open  a  User  Mission 
within  CARVE  (see  Appendix  J,  steps  1-6).  Open  the  Map  (see  Appendix  J, 
step  8).  The  following  waypoint  options  are  available  (Fig.  L-l): 

A.  Add  waypoint 

B.  Delete  waypoint 

C.  Move  waypoint 

D.  Move  spot  report  point 

E.  Map  position  (on/off).  On:  Center  vehicle  on  map;  Off:  Click  on  vehicle  and  move 
around  map 

F.  Zoom 

G.  Return  zoom  to  0% 

H.  Save  map 

I.  Save  waypoints 

J.  Slider  for  zooming  map 


Fig.  L-l  CARVE  waypoint  options 
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2.  Add  Waypoints.  Click  on  the  bull’s-eye  with  the  plus  sign  to  add  waypoints 
(Fig.  L-2).  Then  click  in  the  map.  The  first  waypoint  should  be  close  to  where 
the  vehicle  is  located.  The  mouse  is  now  an  orange  bull’s-eye. 


Fig.  L-2  How  to  add  navigation  waypoints 


3.  Dynamic  Actions.  Every  time  a  waypoint  is  added,  an  Action  screen  will 
appear  (Fig.  L-3).  This  allows  the  user  to  customize  the  waypoints. 


Fig.  L-3  How  to  customize  dynamic  waypoint  actions 
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4.  Stop  Waypoint.  The  Stop  waypoint  stops  the  vehicle  or  robot  in  that  location 
for  a  set  duration  (in  seconds)  (Fig.  L-4). 


Fig.  L-4  How  to  use  the  Stop  waypoint  action 


5.  SetSpeed  Waypoint.  This  waypoint  action  allows  you  to  change  the  speed  of 
the  vehicle  or  robot  at  a  given  location  (in  miles  per  hour)  (Fig.  L-5). 


Fig.  L-5  How  to  use  the  SetSpeed  waypoint  action 
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6.  Finished  Adding  Waypoints.  When  you  have  finished  adding  waypoints  to 
the  map,  click  on  the  bull’s-eye  with  the  plus  sign  to  exit  this  mode  (Fig.  L-6). 


Fig.  L-6  How  to  stop  adding  waypoints 


7.  Adjust  Waypoints.  It  is  possible  to  adjust  the  waypoints  that  were  previously 
placed  on  the  map  by  clicking  on  the  left  mouse  button  and  dragging  the 
waypoint  (Fig.  L-7). 


Fig.  L-7  How  to  adjust  waypoints 
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8.  Adjust  Actions.  It  is  also  possible  to  adjust  or  edit  the  Actions  by  clicking  on 
waypoint  with  the  right  mouse  button  (Fig.  L-8). 


Fig.  L-8  How  to  adjust  waypoint  actions 


9.  Close  Edit  Menu.  To  close  this  menu,  select  the  blue  box  (Fig.  L-9). 


Fig.  L-9  How  to  close  waypoint  edit  menu 
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10.  Save  Waypoints.  Select  the  “save  waypoints”  button  on  left-hand  side  of  the 
screen  (Fig.  L-10).  This  will  open  a  save  window.  Enter  a  unique  file  name 
(e.g.,  WaypointsHRIexperimentl)  and  click  Save.  Write  down  the  file  name 
because  it  will  be  needed  for  step  12. 


Fig.  L-10  How  to  save  waypoints 

1 1 .  Exit  CARVE.  Exit  CARVE  to  set  up  the  application  to  automatically  load  the 
waypoint  file  in  the  future. 
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12.  Setting  up  Automatic  Loading  of  Waypoints.  Right  click  on  the  CARVE 
shortcut,  click  on  properties,  and  add  a  space,  followed  by 
/WaypointsiWaypointHRIexperimentl.txt  in  the  Target  (Fig.  L- 11).  Press 
Apply  and  Ok.  Now  when  you  log  back  into  CARVE,  your  saved  map  and 
waypoints  will  already  be  loaded.  Waypoints  can  be  edited  at  any  time,  just 
remember  to  save. 


Fig.  L-ll  Set  up  automatic  loading  of  waypoints  in  the  CARVE  shortcut  properties 
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Appendix  M.  Directions  to  Set  Up  the  Radar  Task 


Approved  for  public  release;  distribution  is  unlimited. 


139 


1.  Setting  up  the  Radar  Task.  CARVE  is  set  up  to  allow  users  to  accomplish  a 
secondary  task  while  attempting  to  conduct  a  primary  task  such  as  monitoring 
the  unmanned  vehicle.  This  appendix  provides  the  steps  needed  to  set  up  and 
run  the  radar  task.  Once  the  CARVE  application  is  open,  click  on  the  Steering 
Wheel  (a  User  Mission  from  RIVET  is  not  required  during  setup  of  the  radar 
task)  (Fig.  M-l). 


Fig.  M-l  Select  the  Vehicle  Operations  submenu  to  set  up  the  task 

2.  Open  Radar  Menu.  To  open  the  Radar  menu,  click  on  Waypoint  Follow 
and  Save  Waypoints  icons  (Fig.  M-2). 


Fig.  M-2  Open  radar  menu 
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3.  Radar  Run  Setup  Screen.  The  Run  Setup  Screen  GUI  allows  the  researcher  to 
prepare  the  user  for  the  upcoming  data  collection.  As  shown  in  Fig.  M-3,  the 
run  name  (1)  is  entered  along  with  the  path  name  (2)  if  a  secondary  task  will  be 
used  during  the  run.  The  radar  check  box  (4)  must  be  selected  for  the  radar  task 
to  start.  The  safety  zone  (5)  sets  the  radius  from  the  center  of  the  radar  screen. 
Any  blip  that  enters  this  safety  zone  is  then  changed  from  friendly  to  enemy  by 
changing  the  color  from  yellow  to  red.  The  path  name  coincides  with  a  text  file 
that  is  created  by  clicking  on  the  create  path  file  (3). 


Run  Setup  Screen 


1.  Enter  Run  Name 


Enter  Radar  Path  Name 


4j 


Radar  CheckBox 


Create  Path  File 


Save  Waypoint  File 


Waypoint 


Import  Waypoint  File 

Waypoint  Stop  Time 


10  v 


Waypoint 

Load 


Close 


Fig.  M-3  Radar  Run  Setup  Screen  GUI 
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4.  Radar  Path  Creation.  The  button  opens  the  Radar  Path  Creation  Screen  (Fig. 
M-4).  The  X  and  Y  coordinates  are  shown  in  the  list  box  (1).  The  text  file  name 
is  entered  in  the  text  box  (2).  To  enter  the  point,  press  the  Click  on  Radar  point 
button  (3)  and  then  click  inside  the  radar  screen  (3a).  This  will  enter  the  point 
in  (1).  This  process  is  repeated  to  make  a  path.  Upon  the  addition  of  a  second 
and  any  successive  points,  the  path  is  drawn  in  red  inside  the  radar  screen  (3a). 
Once  the  path  is  complete,  enter  a  path  length  in  seconds  (4)  as  well  as  the 
length  of  pause  between  this  path  and  the  next  one  (5).  If  satisfied  with  the  path, 
click  save  path  (6)  and  the  number  of  completed  paths  will  increase  in  the  label 
to  the  right  of  the  save  button.  Once  all  the  paths  are  completed,  click  Ok. 
Clicking  the  Clear  button  will  clear  all  the  data  from  the  screen,  while  the  cancel 
button  will  close  out  the  GUI  without  saving  anything. 


Fig.  M-4  Radar  Path  Creation  Screen  GUI 


Approved  for  public  release;  distribution  is  unlimited. 


142 


5.  Path  Data  Output.  The  default  file  name  entered  is  PathFileName  and  should 
be  changed  by  the  user.  It  is  recommended  that  radar  files  be  named  according 
to  run  name  to  help  with  setup  and  record  keeping.  This  file  is  saved  in  the 
bin\Release\Radar  folder.  Each  file  should  have  a  unique  name.  If  the  path  file 
name  is  the  same  as  a  previous  file,  it  will  overwrite  that  previous  file.  The 
radar  path  text  file  has  X  and  Y  coordinates  on  the  same  line  separated  by  a 
comma.  For  parsing,  each  path  is  separated  by  newPath  (Fig.  M-5).  The 
timeline  is  followed  by  a  series  of  numbers  that  represent  the  time  to  complete 
the  path  followed  by  the  time  between  paths. 
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Fig.  M-5  Sample  data  output  of  radar  paths 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


3-D 

3 -dimensional 

AAR 

after-action  review 

ARIBO 

applied  robotics  for  installations  and  base  operations 

ARL 

US  Army  Research  Laboratory 

BOLT 

basic  operations  layer  transmissions 

CACTF 

combined  arms  collective  training  facility 

CARVE 

Control  of  Autonomous  Robotic  Vehicle  Experiment 

GDRS 

General  Dynamics  Robotic  Systems 

GUI 

graphical  user  interface 

HITL 

human-in-the-loop 

HRI 

human-robot  interaction 

IP 

Internet  protocol 

LAN 

local  area  network 

NPC 

non-player  character 

PIP 

picture-in-picture 

RCTA 

Robotics  Collaborative  Technology  Alliance 

RIVET 

Robotic  Interactive  Visualization  and  Expe: 
Technology 

SQL 

structured  query  language 

TAC-C 

tactical  autonomous  combat  chassis 

TF 

task  force 

TGE 

Torque  Game  Engine 

UAV 

unmanned  air  vehicle 

UDP 

user  datagram  protocol 
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VE  virtual  environment 

WEI  World  Editor  Inspector 
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